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Abstract 


Integrated teaching is teaching in which the educational contents of subjects are interconnected. 
Connections can be discussed in a specialized integrated subjects e.g., Integrated science. 
Integrated science is a subject that connects knowledge from chemistry, biology, physics, 
geography, and geology. One topic is discussed from a different point of view, and students should 
transform their theoretical knowledge in practice. Worksheets can serve as a teaching material 
that can provide help to students as well as teachers and guide students on their path to knowledge. 
But how should a worksheet look like and how should the tasks be designed? This contribution 
describes a suggested method for making a worksheet for integrated (science) teaching that we 
call CCCTER. CCCTER is abbreviation for choose, connect, create, try, evaluate, remade. The 
method implements the national Framework education program for secondary general education 
(grammar schools) in the process of making the tasks. The principles choose, connect, and create 
are demonstrated. It provides a sample task of a worksheet for integrated (science) teaching. 
Keywords: methodology for creating worksheets, integrated science, integrated teaching 


Introduction 


Integrated teaching is a term that is not firmly entrenched in the literature. In our 
research we define integrated teaching as teaching in which the educational contents 
of subjects are interconnected “while the integrated educational content follows the 
aims of all integrated subjects. At the same time, a new aim is formed, which results 
from the integrated whole” (RakouSova, 2008, p. 15). “It implements interdisciplinary 
relationships and a connection between theoretical and practical activities” (Pricha et al., 
2001, p. 87). We perceive integrated teaching as The Integrated Model from publication 
Ten ways to Integrate Curriculum written by Robin Fogarty (1991). In this publication 
in The Integrated Model “interdisciplinary topics are rearranged around overlapping 
concepts and emergent patterns and designs. The integration sprouts from within the 
various disciplines, and teachers make matches among them as commonalities emerge” 
(Fogarty, 1991, p. 64). 


This is an open access article under the @ &) 
https://doi.org/10.33225/BalticSTE/2021.07 Creative Commons Attribution 4.0 (c) 
International License BY NC 
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According to the research made in 2020 in the Moravian-Silesian region of the 
Czech Republic (Bartonova & Kriéfalusi, 2020) a total of 66,4 % of secondary school 
teachers, that were part of the survey, think that they do include integrated teaching into 
their lessons. The second most common way of doing it was by modules or topics that 
are included as a part of more subjects. A total of 33.6 % of respondents think that they 
do not include integrated teaching into their lessons and the third most common reason 
for that is (according to the respondents) lack of didactic materials. Also, the research 
shows that the teachers may not understand the term integrated teaching correctly and 
most probably confused it with inter-subjects’ relations. Further research is needed to 
confirm that. 

As well as integrated teaching a worksheet is hard to define by one source. One 
definition is that “a worksheet is a set of tasks, exercises, didactic image material etc., 
that usually serves to independent student practice or provides him with a guide to his 
work” (Capek, 2015, p. 124). The Cambridge dictionary defines a worksheet as “a 
piece of paper with questions and exercises for students”. Another perspective is that “a 
worksheet is a piece of paper, a computer screen, or a projection that contains problems. 
These problems have right and wrong answers, and there is generally only one way to 
complete a problem” (Ransom & Manning, 2013, p. 188). Worksheets “include activities 
which give the students main responsibility in their own learning” (Yildirim et al., 2011, 
p. 45). Furthermore, according to Kurt (Atasoy et al., 2011) the worksheets “are very 
useful to observe and obtain results, can provide opportunity to reach new knowledge by 
way of student’s own efforts, allow students to form hypothesis and do observation and 
experiment about a subject, help teachers to understand students’ cognitive framework, 
can promote learning by doing and living, and keep students on task and make the 
learning fun” (p. 657). 

Another point of view is that “worksheet serves to facilitate student’s understanding 
in learning material by minimizing the role of teachers” (Nurrohmahadita, 2018, p. 142). 
In a pedagogical dictionary a worksheet is not defined but a workbook is. A workbook is 
defined as “a type of cvicebnice”’. Cvicebnice is defined as “type of textbook, the purpose 
of which is to repeat, consolidate certain knowledge, skills, creating habits etc.” (Pricha 
et al., 2001, pp. 174 & 31). According to Ransom and Manning (2013) ,,the workbook is 
usually just a collection of worksheets with a glue on the spine” (p. 188). 

A worksheet is a type of teaching material. Teaching material is “material or 
content that must be mastered by students through learning activities in accordance with 
the desired curriculum, arranged systematically, both written and not so as to create an 
environment or atmosphere that allows students to learn” (Fajriah & Suryaningsih, 2020, 
p. 1). 

According to research (Podolak & Danforth, 2013) students favour the worksheets 
over textbooks and homework in a Modern Physics course. The research of Sulistiyowati 
et al. shows that “the application of STEM-based worksheet could effectively increase the 
science literacy” (Sulistiyowati et al., 2018, p. 89). Students’ worksheets for integrated 
science were made for example by Widodo et al. for issues about motion in humans. 
The worksheets integrate students’ knowledge about biology, physics, and chemistry. 
Students must integrate that knowledge to solve the problem (Widodo et al., 2018). 

“Integrated science lesson is a natural science study whose study materials include 
biology, physics and chemistry as a whole” (Widodo et al., 2018, p. 2). The purpose of 


https://doi.org/10.33225/BalticSTE/2021.07 | | 
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that lessons is “to enable learners to connect the concepts contained in the three different 
disciplines and apply them in the real world experienced by learners in their daily lives” 
(Widodo et al., 2018, p. 2). 

Authors define integrated science lessons as lessons that connect knowledge from 
chemistry, biology, physics, geography, and geology. In those lessons the students think 
about a topic from a different point of view — as a chemist, biologist, physicist, geologist, 
geographer. They should be as close to the real life as possible, and students should 
verify their theoretical knowledge in practice. 

Since one of the reasons for not implementing integrated teaching into lessons is 
the lack of materials, we decided to make the teaching material (worksheets in particular) 
that teachers may use in their integrated (science) lessons. The question is how should a 
worksheet be designed? 

In the first steps of creating a worksheet for integrated science that is discussed 
further, a mind map is a useful tool. By a mind map is meant “activities that work with 
associations” (Capek, 2015, p. 333). “Mind mapping allows students to imagine and 
explore associations between concepts” (Davies, 2010, p. 280). One of the advantages 
of mind mapping is that it “promotes creative thinking” (Davies, 2010, p. 282). On the 
other hand, “types of links being made are limited to simple associations” (Davies, 2010, 
p. 282). When we want to make more complex connections, we may use concept maps. 


Methodology for Creating Worksheets for Integrated Science 


The content of worksheets must be based on the national Framework education 
program (hereinafter referred as FEP) for secondary general education (grammar 
schools). A method “CCCTER” that stands for “choose, connect (FEP with tasks and 
theory with practice), create, try, evaluate, remade” is a suitable method for creating 
worksheets for Integrated science and is further described. 

The worksheet tasks were created by following these steps (choose and connect) 

1) First, we chose one topic that is learned in chemistry. Five subtopics were 
chosen, each subtopic is a theme of one of the worksheets. Then we thought 
about how to teach each subtopic by integrated science teaching based on 
our experience as teachers. A suitable way of doing that was a mind map or a 
concept map. 

2) In that in mind the expected outcomes of all subjects of educational area Man 
and Nature (that includes the following subjects: chemistry, biology, physics, 
geography, geology) as well as of the cross-curricular subject Environmental 
education, that could be connected to the topic, were chosen. 

3) In phase three the chosen expected outcomes were connected to the tasks that 
aim to meet the given expected outcome or more expected outcomes. At least 
one of the tasks must connect theory with practice. 

All worksheets are created according to the following criteria (create) 

1) Each task of a worksheet must be connected to at least one expected outcome 
from all subjects in the educational area Man and Nature. 

2) Each worksheet must contain at least five tasks. Each task is connected at 
least to one of the subjects of educational area Man and Nature (the tasks can 
be interdisciplinary). 


| | https://doi.org/10.33225/BalticSTE/2021.07 
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3) Each worksheet must include laboratory work that can be inquiry based. 

4) Each worksheet must contain such tasks in which there is a practical use of 
inter subject relations and context. 

5) Each worksheet must respect active learning and activating methods of 
teaching because activating methods of teaching “enable the synthesis of 
the findings from various subjects and their usage in practical or didactically 
modified tasks or situations in a unique way” (Novakova, 2014, p. 5). 

6) Each worksheet must contain different types of tasks from Tollingers’ 
taxonomy. 

a. Tasks requiring memorable reproduction of knowledge e.g., what is the 
weight of... (Svec et al., 1996, p. 55) 

b. Tasks requiring simple cognitive operations with knowledge e.g., 
determine the weight of... (Svec et al., 1996, p. 56) 

c. Tasks requiring complex cognitive operations with knowledge e.g., 
make a scheme of... (Svec et al., 1996, p. 56) 

d. Task requiring transmission of knowledge e.g., write brief content of... 
(Svec et al., 1996, p. 57) 

e. Task requiring creative thinking e.g., prove that... (Svec et al., 1996, 
p. 57) 

After the worksheets were created, the following steps proceed (try, evaluate, remade). 

1) Worksheets were given a try in class with a sample of students. The feedback 
from students was given. 

2) Worksheets were given to the experienced teachers for evaluation. The 
feedback was provided by an interview. 

3) The feedback was processed and evaluated. 

4) Worksheets were remade to be more suitable for students as well as for 
teachers. 

5) The process of try, evaluate and remade can be repeated as many times as 
necessary. 


Outcome 
The example of choose, connect, and create put into practice is given. 
Phase one - choose 


First, one topic that is learned in chemistry was chosen — Halogens. Then authors 
reflected about their experience as teachers and thought about how this topic can be 
taught by integrated science learning. A mind map was made (see Figure | part 1 and 
part 2). As shown in the mind map, several connections among subjects can be made. 
For example, when teaching about yperite teachers can combine history (World War 
T) with chemistry (chemical compound) and biology (physiological effects) as well as 
with geography (Ypres, Belgium). Polyvinyl chloride can be thought from a chemist’s 
perspective (chemical compound, polymer) as well as ecologist’s (environmental 
pollution and sustainability). Sodium chloride is an important chemical compound 
(chemistry) with specific properties (physics), that plays an important role in our body 


https://doi.org/10.33225/BalticSTE/2021.07 | | 
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(biology) and is an important component of The Dead Sea (geography). That is why 
topic Halogens is suitable for integrated lesson. 

From the mind map five subtopics emerged. The subtopics were named as Halogens 
and plastics, Halogens and the ozone layer, Halogens and their cycle, Halogens and We, 
Halogens and their structure. After several discussions there was a need to “unchemistry” 
them in their titles so they will be judged by their content and not through their titles. 
New titles of the subtopics were Plastics, Ozone layer, Chlorines’ cycle, Halogens in our 
hands, and Look for structure behind everything. 


Phase two - connect 


The expected outcomes of all subjects of educational area Man and Nature as well 
as of the cross-curricular subject Environmental education, that could be connected to 
these subtopics were chosen and connected with the tasks. An example is given on the 
following task that is part of a subtopic Plastics. 

Expected outcome — Physics: The pupil shall distinguish between scalar and vector 
quantities and employ them while solving physical problems and exercises (Research 
Institute of Education in Prague, 2007, p. 27). 

Expected outcome — Chemistry: The pupil shall characterise the basic groups 
of organic compounds and their significant representatives, evaluate their raw material 
sources, their application in practice and their effect on the environment (Research 
Institute of Education in Prague, 2007, p. 30). 


Phase three - create 


The task introduction: 


obtained. Its calorific value is 22.5 MJ per kilogram. Calorific value is a property of 
fuel that indicates how much energy is released by completely burning one unit, which 


is usually kg. By burning one kilogram of PVC, we get energy of 22.5 MJ. This energy 
can be converted to kWh according to the relation 1 J = 2,778 - 107 kWh. 


One of the ways to dispose of PVC is to burn it. By burning PVC, energy is 





The task: 
For how long will a light bulb light with a use of energy of 0.006 kW per hour for 


which energy will be obtained by burning | kg of PVC? The calorific value of PVC is 
22.5 MJ/kg. 


| | https://doi.org/10.33225/BalticSTE/2021.07 











Proceedings of the 4" International Baltic Symposium on Science and Technology Education, BalticSTE2021 | 


12 


Figure 1a 
Mind Map “Halogens” (Mind map was made using Coggle https://coggle.it/) 
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Figure 1b 
Mind Map “Halogens” (Mind map was made using Coggle https://coggle.it/) 
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Conclusions and Implications 


Based on previous research one of the main reasons for not implementing 
integrated teaching into lessons is a lack of suitable teaching materials. In this contribution 
new method that can be used for making integrated worksheets for all subjects of 
educational area Man and Nature is described with a practical example. A united method 
for making worksheets for integrated (science) teaching is important so a database of 
worksheets for integrated (science) teaching can be made and one of the main obstacles 
for not implementing integrated teaching into lessons (lack of suitable materials) can be 
overcome. 
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Abstract 


This research deals with a possible use of history of mathematics in mathematics education. In 
particular, history can be a fundamental element for the introduction of the concept of integral 
through a problem-centred and intuitive approach. Therefore, what follows is dedicated to the 
teaching of mathematics in the last years of secondary schools, where infinitesimal calculus is 
addressed. The thesis here proposed is that the resort to Archimedes’ use of exhaustion method 
and to Newton 8 initial lemmas expounded in his Principia Mathematica are useful means to reach 
a genetic comprehension of the concept of integral. Hence, two demonstrations by Archimedes 
and two lemmas by Newton are used to prove such thesis. A further idea here proposed is that 
history of mathematics can be of help for an interdisciplinary education. 

Keywords: interdisciplinary education, mathematics education, science history, secondary 
schools 


Introduction 


This research concerns the possible use of the history of mathematics within 
mathematical education in the last two years of the secondary schools because it regards 
the teaching of integral calculus. 

History of mathematics can be a useful element in mathematics education because: 

1) starting from history, the pupils can understand the conceptual origin of the 
mathematical concepts and to realize that they have been created to solve «concrete» 
mathematical problems. The formalization of the concepts is a successive step, when the 
numbers and the width of problems increased, and the necessity of precise definitions 
became necessary. However, if the teaching begins with the formal apparatus there is the 
risk that the learners do not guess its scope and utility and mistake the real meaning of 
the formalization. 

2) A historical approach is useful for the students to realize the human aspect 
of mathematics. Mathematics exists thank to the hard work of the mathematicians. 
It is a human activity in which before reaching the final presentation of a concept or 
solution of a problem, a long history of failures, attempts and improvements has been 
needed. Mathematics, if appropriately presented is neither sterile nor mechanic, though, 
obviously, several technicalities have to be learned. 

3) Nowadays «interdisciplinarity» is a common place, but to conceive an, at 
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least partially, interdisciplinary education is not easy. History of mathematics can be 
of help. We will speak of Archimedes and Newton in reference to the birth of integral 
calculus. It might be possible to conceive an educational itinerary in which the teachers 
of history, physics, mathematics, and history of philosophy (where this discipline is 
taught) collaborate. For example, in the programme of history, it would be possible 
to stress the aspects connected to history of science and in the programme of physics 
the genetic problems which determined the introduction of certain concepts might be 
highlighted. In this way, an education based both on the connection of different subjects 
and on the understanding of the conceptual grounds of each specific discipline could be 
implemented. 


Area of Curvilinear Figures and Integral Calculus 


The example I am proposing concerns the calculation of curvilinear and 
mystilinear areas. Today the integral calculus and the concept of definite integral is used. 
The integral calculus is taught in the secondary schools after the formal introduction of 
the concepts of function, of limit and of the differential calculus. I am not claiming this 
educative methodology to be incorrect. Rather, a different one will be proposed for the 
teachers to have diverse options to follow in the difficult enterprise to make mathematics 
an interesting subject for most learners. The teachers themselves will be able to decide, 
according to pupils’ level and propensities, the most appropriate approach. 

Thus, before introducing the formal concepts of infinitesimal calculus, it would 
be possible to proceed like this: introduce only the concept of function. After that, 
analyse the way in which the problem of the calculation of curvilinear areas was faced 
by Archimedes (287-212 B.C.). This will offer the possibility: 


a) to understand what the concept of limit is and to grasp that it is necessary while 
dealing with the areas of curvilinear figure. 

b) To guess the profound meaning of the proves ad absurdum as well as the 
nuances and the subtleties behind such difficult problems as those proposed. 

c) to comprehend the greatness and limits of Archimedes’ approach as well as 
the necessity to overcome such an approach if you intend to achieve a general 
treatment of curvilinear areas. Therefore, Newton’s concept of integral will be 
introduced. 

d) Finally, after that the learners will have understood all the conceptual and 
computational bases of the infinitesimal calculus, a more formal approach will 
be proposed. But now the learners will fully grasp the conceptual meaning of 
such a formalization. 


Archimedes and the Exhaustion Method: First Example 


The procedures of Archimedes are extremely interesting because they seem 
an «anticipation» of the integral calculus, at least from a conceptual, if not operative, 
standpoint. 

In fact, Archimedes invented two methods correlated to the calculations of 
curvilinear surfaces and volumes: the method of barycentres and the method of 
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exhaustion. We will focus on two examples of the latter. The method of exhaustion was 
invented by Eudoxus of Cnidus (first half of the 4th century BC) and it is also used in 
the last books of Euclid’s Elements (Euclid 2008, Book XII, Propositions 2, 5, 10, 11, 
12, 18, pp. 473-475, 480-481, 486-495, 503-504). However, Archimedes was the true 
master of this method. 

It works like this: you want to prove that two quantities A and B (areas or volume 
of two figures) are equal. Suppose they are not and be A>B. You construct a series of 
figures which approximate the figure A by defect and among them you find one whose 
difference from A is less than the supposed difference A-B. Then, you are able to prove 
that this figure should be, at the same time, greater and less than B, which is absurd. 

Afterwards, you suppose A<B and, in a similar manner, you show a figure which 
should be bigger and less than B. Absurd. Since A cannot be either greater of less than 
B, it is equal. 

The area of the circle. From the first years of the secondary schools, the pupils 
know that, given a circle whose radius is r, its surface is 17°. However, if you ask them 
to prove this statement, though in an intuitive manner, probably no one will answer the 
question, because the answer is not easy (try to do this «experiment»). 

The first mathematician to solve such a problem was Archimedes in his work 
Measurement of the circle (Archimedes 2010, pp. 91-98). He developed the following 
brilliant reasoning to prove that «Each circle has the same surface as the right triangle 
of which a leg (cathetus) is equal to the radius and the other leg to the circumference». 


Figure 1 
The Diagram Drawn by Archimedes to Determine the Circle's Area 





Note: Adapted from Archimede 1974, p. 226. The editor uses Latin rather than Greek letters to indicate 


points 
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Archimedes proceeds like this (Fig. 1): suppose the circle ABCD to have an area S 
greater than £. It is possible to inscribe the square ABCD of area Q in the circle. Suppose 
OX<E or, as Archimedes would say, the area of the four circular segments AD, DC, CB, 
BA is bigger than the difference by which the area of the circle exceeds the area of the 
triangle E. Bisect the four arcs of circumference AD, DC, CB, BA. Be F and M two of 
such points of bisection. Let us construct the regular octagon FA, AM, MD, ..., whose 
area is O. Suppose O>E, or, as Archimedes wrote, that the sum of the areas of the eight 
circular segments FA, AM, MD, ... is less than the difference by which the area of the 
circle exceeds the area of the triangle E (if you suppose O<E, it is possible to continue 
bisecting the arcs of the circumference until reaching a polygon of 2” sides whose area is 
greater than £, but less than S). From the centre N of the circle, draw the perpendicular 
NO to the side AF of the octagon. Such a segment is less than the altitude of the triangle, 
which is equal to the radius. The octagon’s perimeter is less of the rectified circumference. 

Therefore, the octagon’s area is less than £. But this is absurd because we have 
supposed O>E. Therefore, the area of the circle cannot be greater than that of the triangle. 

Through a similar reasoning it is possible to prove that the area of the circle cannot 
be less than that of the triangle. 

Commentary to use in an educational perspective. The just expounded 
demonstration gives rise to several comments which are very instructive. Some of 
them concern the internal logic of the proof; others are relative to the concepts used 
by Archimedes in comparison to modern notions. Let us begin with the first kind of 
comments: 

a) In the proof there is a critical point: Archimedes seems to give for granted that 
the perimeter of the polygon inscribed in the circumference is less than the circumference 
itself. In fact, this problem does not exist because in the assumptions of his work On the 
sphere and cylinder Archimedes writes as first Assumption: “Of all lines which have the 
same extremities the straight line is the least” (Archimedes 2010, p. 3). 

b) Further possible problem: it might happen that the difference between the area 
of the circle and that of the inscribed polygons diminishes but without tending to 0. 
Archimedes considered this problem, too. In propositions 3-6 of the first book of On the 
sphere and cylinder, he proved to be possible to inscribe in or circumscribe to a circle a 
series of regular polygons whose area has any ratio (or difference) with the area of the 
circle (Archimedes 2010, pp. 6-10). 

Now let us come to commentaries which are useful not only to compare the 
modern concepts with those used by Archimedes, but also to introduce the learners 
to such concepts showing that their necessity derives from “concrete” mathematical 
problems as the one we are analysing. I am specifically referring to the notion of limit: it 
is usually introduced as the limit of a function. Given a function /;R-R and a point x, of 
its domain you say that f(x) = /, if for any real number € > 0 , a real number 6 exists such 
that, if | x —x,|<6, then | f(x) —f(x,)| < €. This definition is easily adaptable to the cases 
in which x tends to infinity or / is infinite. 

Now, what is present in Archimedes’ proof is the limit of a series of figure of 
which we consider the function-area, which can be reduced to a real function. To be 
more precise, Archimedes - to transcribe his idea in modern terms - considers as domain 
the perimeters of the regular polygons whose number of sides is of the form 2”, also 
including the circumference — let us name P such a set — and, after that, the areas of 
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each of such figures, also including the area of the circle — let us name S such a set — 
regarded as functions of the perimeters. The perimeter and the areas are real numbers; 
therefore, the function-area f can be represented as : R- R . Be P, the length of the 
circumference and P the perimeter of any inscribed polygon, so that f(P,) is the area of 
the circle and f (P) is the area of the regular polygon whose perimeter is P. Thus, in the 
reasoning of Archimedes it is implied that, for any real number ¢ > 0, a real number 6 
exists such that, if |P— P,| <6, then | f(P) —f(P,)| < €. This is a simple transcription in 
a modern and relatively formalised language of what exists in Archimedes. There is no 
interpretative stretch. Hence, from an educative point of view, it is to highlight that this 
easy proof offered by Archimedes seems a natural way to introduce the concept of limit 
of a function; before in an intuitive manner — as the teacher might simply say that, when 
the difference between the perimeters of the inscribed polygons and the circumference 
becomes smaller and smaller, the same happens with the differences between the area of 
the polygons and that of the circle — and, afterwards, in a more formalised way. 

On the other hand, it is appropriate to show the pupils a characteristic of Archimedes 
proof: the concept of limit is not used to develop a calculation through which to compute 
by means of formal steps, which might be repeatable, mutatis mutandis, to solve other 
problems. Rather, it is used to develop an ad absurdum reasoning which works because 
of the specific characteristics of the regular polygons inscribed in a circle. Therefore, 
while, in the genial Archimedes’ procedure the general concept of limit is present, a 
general method to calculate the areas of, at least, a class of curvilinear figures is missing. 
As we will see, there are “ingredients” and ideas which can be used in other cases, but a 
general method of areas calculus lacks. Therefore, the students will perfectly understand 
the need to develop a method more general than Archimedes’. 

An interesting and very instructive consideration concerns the way in which 
Archimedes used the ad absurdum reasoning. In this kind of proves, the assumption the 
theorem to be false (in particular, the assumption that the area of the triangle E is less 
than that of the circle) implies a proposition p and its negation not-p to be true. In our case 
the proposition p is: «the area of the octagon is bigger that the area of the triangle». The 
proposition not-p is not a proposition which is true because it is an axiom or a theorem 
previously demonstrated, as it happens usually in the ad absurdum proves. Rather, the 
propositions p and not-p are both deduced during the argumentation. In a sense, in this 
proof, and in general, in the proves for exhaustion, the hypothesis the theorem to be false 
destroys itself. There is not a contradiction with an already established truth. This is a 
particular of the exhaustion method worth being stressed to the pupils. 


Archimedes and the Exhaustion Method: Second Example 


It is appropriate to present a further example of the way in which the exhaustion 
method can be applied because the conceptual and didactical considerations which can 
be drawn hold a remarkable educative value. 

The theorem whose prove will be proposed concerns one of the most famous 
results obtained by Archimedes: the surface of every sphere is the quadruple of its great 
circle. Namely, if 7 is the radius of the sphere, its surface is 4277. This statement is 
proved by Archimedes in On the Sphere and Cylinder as Proposition 33 of the first book 
(Archimedes 2010, pp. 39-41). Five other theorems proved by Archimedes in the same 
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work are necessary along the demonstration. For convenience of the reader, I mention 
them: 
1) Theorem I: Given two unequal magnitudes A and B, with A>B, it is possible 
to find two segments, whose lengths are resp. S and s such that S: s < A: B. 
(On the Sphere and Cylinder, I, 2, Archimedes 2010, p. 5). 
2) Theorem II: Given two unequal magnitudes A and B, with A>B and a circle, it 
is possible to inscribe a regular polygon of side / and to circumscribe a regular 
polygon of side L, so that L: 1 <A: B. (Ibidem, 1,3, pp. 6-7). 
3) Theorem II: Consider a regular polygon ABCM... whose number of sides 
is of the form 4n, being n a natural number, inscribed in a great circle of a 
sphere. Rotate this polygon around the sphere’s diameter 4A’, then the area 
of the figure obtained by a complete rotation is less than four times the area 
of the sphere’s great circle (Ibidem, I, 25, pp. 32-33). 


Figure 2 
The Diagram Used by Archimedes to Prove Proposition 25, book I, On the Sphere 
and Cylinder 








Note: Retrieved from Archimedes 2010, p. 32. The editor uses Latin rather than Greek letters to indicate 
points. 


4) Theorem IV: If the regular polygon of 4n sides is circumscribed to a great 
circle of the sphere, and it rotates around a sphere’s diameter in a manner 
similar as the polygon of the previous theorem, the area of the figure obtained 
by a complete rotation is bigger than the area of the sphere (/bidem, I, 28, p. 
35-36). 

5) Theorem V: Consider the figure of rotation constructed in the two previous 
theorems: the areas of the circumscribed and of the inscribed figures are as 
the squares of the sides’ ratio of the polygon circumscribed to the great circle 
of the sphere to that inscribed. The volumes of the two figures are as the cubes 
of the sides (Ibidem, I, 32, pp. 38-39) 
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Figure 3 
The Diagram Used by Archimedes to Prove Proposition 32, Book I, On the Sphere 
and Cylinder 





Note: Retrieved from Archimedes 2010, p. 38. The editor uses Latin rather than Greek letters to indicate 
points. 


Let us not see Archimedes’ proof of Proposition 33: be S a sphere and A a circle 
equivalent to four time the sphere’s great circle. It is required to prove that the surfaces 
s(S) of S and s(A) of A are equal. Suppose, ad absurdum, s(5), > s(4). Under this 


hypothesis, it is possible to choose two segments B and C such that ¢ = (Theorem I). 
Be D the mean proportional between B and C, namely B: C = D: C. Be the sphere cut by 
a plane through its centre along a great circle EFGH. Inscribe in and circumscribe to the 
sphere two similar polygons with a number 47 of sides, under the condition that between 
the side P of that circumscribed and the side p of that inscribed the relation P: p < B: D 
(Theorem IT) holds. Consider the figures generated as in Theorem HI and IV. Indicate 
the surface of that inscribed as s(/) and the surface of that circumscribed as s(C). Then 
s(C): s(@Z) < s(S): s(A), which can be deduced by the following series of steps: 


I. s(C): sD) = P?: p? (Theorem V). 

Il. P:p<B:D (Assumption). 

Il. B:C < s(S): s(A) (Assumption). 

IV. B:D=D:C (Assumption). 

V. B: C=B?: D’ (Direct consequence of IV). 
Vi. s(S): s(A) > B?: D? (From II and V). 

Vil. Be DP > Pp? (From II). 

VIL. s(C): s(Z) < s(S)-s(A) (From I, VI and VII). 





https://doi.org/10.33225/BalticSTE/2021.16 | | 





Proceedings of the 4" International Baltic Symposium on Science and Technology Education, BalticSTE2021 








23 


However, this is impossible because s(C) > s(S) (Theorem IV) and s(/) < s(A). 
This absurd conclusion implies that the area of the sphere cannot be bigger than four 
times the area of its great circle. Through an analogous reasoning, Archimedes proved 
that it cannot be even smaller. Thence, it is equal. 


Figure 4 
The Diagram Used by Archimedes to Prove Proposition 33, Book I, On the Sphere 
and Cylinder 





Note: Retrieved from Archimede 2004, p. 154. The editor uses Latin rather than Greek letters to indicate 
points 


Commentary to use in an educational perspective. This proof offers many reasons of 
interest which can represent useful ideas to develop from an educative standpoint. They 
concern the internal logic of the proof and its level of generality. Let us begin with the 
first question. 

Logic of the proof: in the demonstration concerning the area of the circle, 
Archimedes used the difference between the area of the circle itself and the area of the 
inscribed (circumscribed) regular polygons of 2” sides. The area of those inscribed (the 
case we have analysed) approximate with an arbitrary exactness the area of the circle. 

In this second proof, the idea is to resort to ratios rather than to differences. One 
might say that the space between the areas of the circumscribed and the inscribed figures 
include the area of the sphere the more precisely the more the number of the sides of 
the polygons inscribed and circumscribed to a great circle of the sphere increases. In 
particular, to prove ab absurdum that the ratio x» cannot be greater than 1, Archimedes 
used the fact that the ratio “2>1 and that “2>*. Thus, his ad absurdum construction can 
be represented like this: 
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Figure 5 
Diagram which Summarizes Archimedes’ Reasoning. Adapted from Archimede 1974, 
p. 152 


s(SVs(4)  s(C/s(D) 


—— ee 








s(S) 
During the prove he demonstrated that, supposing 3,4) > 1 you reach a situation 
h s(C) s(S) hichies ‘hI 
where 2) S ay° Which is impossible. 


Where is the concept of limit here involved? It is involved in the construction 
which exploits theorem III because, if you indicate by | + ¢ the ratio A/B, being an 
arbitrary real number (but, of course, we are interested in the case in which ¢ is “very 
small), this theorem states that Z:/<1 + ¢, namely that it tends to 1, which, in its turn, 


s(C) 
implies that the ratio between the areas 3 can differ from | by any arbitrary small 
: . . c s(S) 
value. This allows Archimedes to insert, ad absurdum, pal between | and Ha)" Thus, 


the concept of limit is implied in this demonstration and is in every proof by exhaustion. 

We have seen two different applications of the exhaustion method, one based on 
the difference and one based on the ratio between figures. The latter application is more 
complex than the former because in the former only two figures are used in the proof (the 
circle and the series of the inscribed polygons, in the part of the proof here proposed; the 
circle and the series of the circumscribed polygons in the other part), whereas in the latter 
three figures are used at the same time: the series of circumscribed figures, the sphere, 
and the series of inscribed figures. 

The first form of the exhaustion method can be called by approximation, the second 
form by compression based on a ratio. There is a third form, namely the compression 
based on a difference, but it is not necessary to enter further details for the aims of this 
research (on this subject see the seminal, pp. 130-133). 

These logical considerations here presented are rather subtle and can be extremely 
useful to develop the logical capabilities of the learners. Thus, it is advisable that the 
teachers develop them in front of the learners. This will be instructive. Let us move on 
some more general reflections. 

General reflections on Archimedes’ method. Archimedes proved several theorems 
by exhaustion. The two demonstrations we have seen are paradigmatic of some general 
features which connote all the proves by exhaustion: 

a) The concept of limit is present and well conceived. Archimedes guessed that 
the only way to calculate, in general, the areas and the volumes of curvilinear 
or mystilinear figures was to approximate them with an infinite series of 
rectilinear figures, or, at least, of figures whose areas or volumes were already 
known. 

b) What expounded in the item a) is sufficient to define that of Archimedes a 
method, in the sense that each prove by exhaustion is based on some leading 
ideas which are always the same. On the other hand, the exhaustion method is 
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not a systematic procedure which can be applied always in the same manner 
to calculate areas and volumes. For every figure Archimedes analysed, some 
specific theorems and constructions are necessary. They are not logically 
deducible from the theorems and constructions applied in different cases. 
Thus, the exhaustion method is not a mechanical procedure based on constant 
and invariable steps. Though a series of general ideas exist, each case requires 
some different specific reasonings. 

c) Furthermore, Archimedes, as it is typical of the tradition of Greek mathematics, 
does not offer the numerical value of the required area or volume, but proves 
the identity of an unknown area with a known area. It is, however, to point out 
that in no works of the Greeks mathematics you find explicit formulas such as 
“area of a triangle is equal to basis by altitude”. In Euclid’s E/ements there are 
propositions (See, for instance Euclid 2008, I, 36, pp. 37-38 for the area of a 
parallelogram and Euclid 2008, I, 37-I, 40 pp. 38-41 for the area of a triangle) 
from which it results clearly that the Greeks had reached many elementary 
formulas for areas and volume, but such formulas are not explicitly stated. 

d) The exhaustion method is not a heuristic method; it is a demonstrative 
method, which does not allow you to calculate the value of a certain area or 
volume if you do not know it; rather, it permits to prove an area or volume to 
be equal to another area or volume if you have reached this result otherwise. 
This is understandable if you think that we are speaking of an ab absurdum 
procedure. 

e) Finally, a brief historiographic note has to be added: many important historians 
of science thought that Archimedes’ method of exhaustion contains most of the 
elements typical of the integral calculus. Heath, albeit with many reservations 
and specifications on the use of the infinite in Greek mathematics, basically 
thought that Archimedes’ exhaustion method “anticipated” integral calculus 
(Heath in Archimedes 2010, pp. CKLU-CLIV). Many other scholars followed 
his opinion. In contrast to this view, far more recently, Napolitani and Saito 
have underlined the fundamental differences between Archimedes’ procedure 
and infinitesimal calculus. According to their train of thoughts, there is no 
generality in Archimedes’ exhaustion proves (see, i.e., Napolitani 2001, Saito 
2013). My opinion, as also clarified in these pages, is in between the two: the 
concept of limit is present in Archimedes in almost modern form, but nothing 
as a calculation of limits, a necessary — though not yet sufficient - ingredient 
for the integral calculus, exists (see also Bussotti 2019). I point out that, in 
this item, I have given only a very slight idea (the aim of this research is 
not historical, but educative) of a complex and articulated historiographical 
debate, with a great number of protagonists. 


Once described this fascinating picture, which can arouse the interest and the 
curiosity of the pupils, the reason to look for a general method which will permit to 
calculate directly the numerical value of curvilinear areas and volumes will be evident. 
The teachers, at this point, might ask the pupils which elements of the Archimedes’ 
method they think to be necessary in order to develop a general procedure and which 
elements have, instead, to be modified. The result of this discussion should be that the 
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notion of limit has to be conserved as well as the idea to approximate the curvilinear 
areas and volumes with known areas and volumes. It is clear that the next step will be 
to better clarify the techniques of approximation. The aim is to transform Archimedes’ 
use of approximation, which is genial and correct, but which needs a new different idea 
for any application, into a systematic procedure. This process will bring to the concept 
of integral. In this manner, the learners will understand profoundly the problem behind 
mathematics, which will, hence, result to be a conceptually rich discipline and not 
merely a set of techniques to learn by heart. The best way to introduce this further step 
is Newton’s work. 


Newton and the Concept of Integral 


Newton dedicated several of his works to the concept of integral and to the integral 
calculus. However, for our aims, it is sufficient to analyse the lemmas on the concept of 
integral included by Newton in the first section of his Mathematical Principles of Natural 
Philosophy (1687 first edition, 1713, second edition, 1726 third edition; see Newton 
1999), which is entitled “method of first and last ratios”. Here the concept of limit is 
used in an almost modern manner and more explicitly than in Archimedes, though its 
formalization in g-6 terms was given only in the 1820s. Newton’s text opens with the 
celebrate “Lemma I’, which states: 


“Quantities, and also ratios of quantities, which in any finite time constantly tend to 
equality, and which before the end of that time approach so close to one another that their 
difference is less than any given quantity d, become ultimately equal”. 

Proof: “If you deny this, let them become ultimately unequal, and let their ultimate 
difference be D. Then they cannot approach so close to equality that their difference is less 
than the given difference d, contrary to the hypothesis”. (Newton 1999, p. 433). 


Two considerations can be proposed in an educative context: 

1) Newton considers two quantities or two ratios of quantities whose difference 
tends to become smaller than any given d and claims their limit to be 0. This 
is the meaning of the expression that the two quantities or ratio of quantities 
“become ultimately equal”. What Newton indicates with d has the same 
meaning as ¢ in the formal definition of limit. However, Newton proceeds 
in a manner which is slightly different from ours: he considers the limit of 
two quantities which are both functions of the time. In practice, Newton 


is claiming that if, being g(t) and /(¢) functions of the time ¢ and if lim,_,; 

g(t) — f (2) = 0 (or equivalently if |t — ¢,| < 6, then |g(4) — f (O| < € or d, as 
Newton writes) the values of the two functions can be considered equal in ¢,, 
which is absolutely true if g and fare continuous in ¢,, as Newton implicitly 
presupposes. In an educative perspective, it is important to remark that the 
reference to the concept of function is a way to grasp Newton’s ideas and to 
transcribe them in a language the pupils know and which creates, hence, no 
problem. However, properly speaking, Newton did not have the concept of 
function. For, he spoke of “quantities” which is a very generic expression that 
cannot be identified with our formal notion of function. This notwithstanding, 
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the concept of limit can subsist and correctly used also in a mathematics in 
which the notion of function is not exactly stated, though we refer always the 
limit to the functions. Only as a consequence to solve mathematical problems 
for which the concepts had to be defined in a more perspicuous manner, we 
reached the formal definition of basic notions such as function and limit of a 
function. 

2) Newton sees the concept of limit in a dynamical, a physical manner. For, the 
independent value is time and, hence, the functions g(f) and f(t) represent, at 
least in principle, physical quantities. It is intuitive that they might represent 
spaces expressed in function of time, or velocities, or accelerations varying 
in the time. It will be instructive to show that, at the dawn of modern science, 
the concept of limit was presented not in terms of a generic and abstract 
independent variable, but in terms of the variable Newton assumed as the 
ground of his physics: time. In spite of the fact that time is a specific variable, 
Newton’s procedure has a general value, whatever the independent variable 
be. 


Let us now see how Newton introduced the concept of integral. 


Lemma II: «If in any figure AacE, comprehended by the straight lines Aa and 
AE and the curve acE, any number of parallelograms Ab, EC, Cd, ... are inscribed upon 
equal bases AB, BC, CD, ... and have sides Bb, Cc, Dd, . . . parallel to the side Aa of 
the figure; and if the parallelograms aKbl, bLwcm, cMdn, ... are completed; if then the 
width of these parallelograms is diminished and their number increased indefinitely, I 
say that the ultimate ratios which the inscribed figure AKbLcMdgqD, the circumscribed 
figure AalbmcndoE, and the curvilinear figure AabcdE have to one another are ratios of 
equality». 

Proof: «For the difference of the inscribed and circumscribed figures is the sum 
of the parallelograms K/, Lm, Mn, and Do, that is (because they all have equal bases), 
the rectangle having as base Kb (the base of one of them) and as altitude Aa (the sum 
of the altitudes), that is, the rectangle AB/a. But this rectangle, because its width AB is 
diminished indefinitely, becomes less than any given rectangle. Therefore (by Lemma 
1) the inscribed figure and the circumscribed figure and, all the more, the intermediate 
curvilinear figure become ultimately equal». 
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Figure 6 
The Diagram Used by Newton to Prove Lemma II. 


af 
om 





Note: Retrieved from Newton, 1726, p. 28 


In this Lemma the concept of integral is clearly stated. For, the teacher, relying 
upon clear and intuitive Newton’s explanations might give a first visual idea of what 
the integral of a function is as follows: you have to calculate the area of the curvilinear 
region included between the graphic of the drawn function and the x-axis (Fig. 7) in the 
interval (a,b). Such interval is divided into a certain number of partitions. A series of 
rectangles is drawn whose sum approximates by defect the searched area and a series 
of rectangles whose sum approximate by excess. Under appropriate conditions, when 
the interval x(z)-x(i+1) tends to 0 and the number of the infinitesimal rectangles tend 
to infinity, the sum of the series of the rectangles which approximate by defect and the 
series of that approximating by excess tend to the same value. Such a value is assumed 
as the required area. 
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Figure 7 
A Figure which can be Used to Give an Intuitive Idea of the Concept of Integral 


a=2% ro r3 r4 r= b 


All these steps are already present in Newton, thought in a less formalised 
manner than ours because the concept of function was not yet defined and that of limit 
was, consequently, conceived in a manner slightly different from ours. But all the 
fundamental ingredients of the integral calculus exist in Newton. Starting from the work 
of this great scientist it is, hence, possible to propose an intuitive, but, however, rather 
precise approach to the notion of integral. Here, you have a positive idea which can be 
developed into a technique or a series of techniques to calculate curvilinear areas and 
volumes. By the way, Newton himself developed many of such techniques. The stage in 
which the infinitesimal concepts are used as demonstrative means to which no technique 
of calculus was associated has been overcome, thanks to Newtons’ idea to consider the 
area of a curvilinear figure like in the Lemma II. 

Newton introduced two other lemmas on the integral calculus, the Lemma III, and 
Lemma IV. The former is particularly worth being mentioned: Newton shows that the 
same conclusions drawn in Lemma II hold if not all the bases of the parallelograms are 
equal, granted that all decrease at infinity. Some corollaries follow, the most interesting 
of which is the fourth one, which, from our standpoint is remarkable because Newton 
claims that the figure obtained increasing more and more the number of the majorant 
and the minorant parallelograms and whose perimeter is AcE (Fig. 6) is not anymore a 
rectilinear figure, but the curvilinear limit of a series of rectilinear figures. In this context 
he used explicitly the Latin term /imites (plural), that is limits, which means that, also 
from a terminological point of view, his conceptions are similar to ours, or better ours 
are similar to his! 

At this point, once understood the fundamental characteristics of the concept of 
integral thanks to Newton’s work, the pupils will also realize that a formal definition of 
function, limit and integral will be necessary to fully guess to what extend the concept 
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and the techniques of integration developed by Newton can be extended. Now the entire 
preparatory work to introduce the notion of limit and of integral in a genetic, problem- 
centred and intuitive way is finished and the learners will be ready to accept more formal 
definitions, which will be easy to be understood after this conceptual introductory work. 


Conclusions and Implications 


The effort of this research has been to show in the concrete case of integral calculus 
the way in which history of mathematics could be used in an educative context. We have 
seen the advantages of a historical approach insofar as it allows the pupils to understand 
not only the chronological origin of the concepts, but also the genetic-conceptual 
one. Through this approach, they will comprehend the progressive constructive steps 
through which the edifice of mathematics has been built. The advantage of this educative 
standpoint relies also upon the possibility to develop an interdisciplinary approach with 
other subjects such as physics, history and history of philosophy. Furthermore, the 
teacher has a great freedom to choose the historical itinerary he prefers. For example, in 
reference to integral calculus other authors different from Archimedes and Newton might 
have been chosen. The didactical itinerary, if well conceived, would have been equally 
valid. It is important to show how the educative path here proposed might be continued 
so as to include the differential calculus: for, following Leibniz’s train of thoughts, it 
will be possible to show that the operations of drawing the tangent to a curve and of 
identifying the points of maxima and minima are opposite to that of calculating an area. 
This is very interesting because it will allow the pupils to intuitively understand what 
today we call the fundamental theorem of integral calculus, that is (to simplify) the fact 
that derivation and integration are two opposite operations. Such result will be achieved 
through the analysis of concrete problems (calculations of areas, of tangents and of 
maxima and minima), which could represent a stimulus for the pupils to increase their 
interest towards mathematics. 

For all the reasons expounded in this research, a historical approach to mathematics 
education is highly recommendable. 
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Abstract 


A new didactical approach named "Learning by Being" (LBB) is proposed and its correlation 
with current educational paradigms in science teaching is analysed. The key idea in LBB is the 
assumption by the students of cognitive goals, and three components are mandatory in LBB: a) 
students personal learning effort, b) student — teacher mutual feedback and c) metacognition. In 
other words, the ownership of cognitive goals and students’ deep intrinsic motivation. Several 
didactic approaches, used within LBB, are analysed: independent research that has an impact 
factor on cognitive achievement equal to 83%; knowledge of success criteria (impact factor — 
113%); ability to reveal similarities and patterns (impact factor — 132%). The core of LBB is 
guided learning effort that corresponds to the notion of teacher—student harmonic oscillator when 
both things — guidance from teacher and students effort — are equally important. 

Keywords: conceptual understanding, learning by being, ownership of cognitive goals, science 
learning. 


Introduction 


The problem of low students’ motivation and interest for learning effort is one 
of most important in nowadays school. The majority of solutions for overcoming 
low motivation come from the constructivist area: learning by doing, active learning, 
learning by understanding, inquiry-based science education, problem— and project — 
based learning, etc. All these approaches differ in the degree of independence in learning. 
The higher is the degree the higher is impact of particular didactic strategy on academic 
achievement of school students (Hattie, 2017). In this way, increasing the students’ 
independence degree in learning is the solution if we want to not only remove boredom 
but also achieve academic success of students. 

This independence in learning should go until it will reach the student’s ownership 
on learning goals. It means that the students both understand their way of learning and 
assume the didactic objectives as their own cognitive goals. It is about metacognition 
(Scharff et al., 2017). Thus, teachers’ didactic goals should become students’ learning 
objectives. This fact corresponds to the results of the theory of visible teaching and 
learning (Hattie, 2009). In order to obtain the assumption by the students of cognitive 
goals, the communication in the frame of the lesson should be based on mutual student — 
teacher feedback and the students’ empowerment through metacognition (Millis, 2016). 

In such a way, a modern didactic strategy contains three pillars: students’ learning 
effort, enhanced student — teacher communication and metacognition (Calalb, 2020). 
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Considering these prerequisites, this short communication analyses several 
moments: the evolution of educational paradigms from /earning by doing to learning by 
being, the correlation of students’ learning effort with metacognition, the main features 
of multilateral student — teacher feedback (feedback from and to all students) and gives 
practical recommendations for applying learning by being concept in inquiry — based 
classroom. 


From Learning by Doing to Learning by Being 


The first step in the ladder of modern educational paradigms is /earning by doing 
(LBD). For example, a form of LBD is ludic education. According to the results of the 
theory of visible teaching and learning (VTL), the impact factor on academic achievement 
of students for ludic education is 35% (Hattie, 2009). Compared with the benchmark 
level, which is equal to 40%, when an experienced teacher applies permanently the same 
conventional strategy during two academic years, LBD has a negative impact factor. 
These numbers demonstrate once again that school does not belong to entertainment 
industry and conventional teaching remains a solid option in modern times. 
The second step in the mentioned ladder is /earning by understanding (LBU). A 
well-known example of LBU is inquiry-based science education (IBSE) (Harlen, 2010), 
which has an impact factor equal to 77%. Thus, the effect of understanding is twice 
higher than the one of doing. The level or complexity degree of communication makes 
the difference between LBD and LBU. The communication should be seen as leverage 
for enhancing students’ involvement degree in classroom activities. 
The next step would be learning by being (LBB) when the students not only 
understand but also assume the learning objectives. In this way, within LBB, the students 
are the subjects of their learning process, construct their cognitive path, and form in 
class sustainable lifelong learning skills. It is about the students’ ownership on cognitive 
goals (Calalb, 2019). Here we will present several characteristic features of LBB. For 
example: 
e The ability of students to carry out independent research, in other words 
guided self-scaffolding, with an impact factor equal to 83%; 

e Knowledge of success criteria or even more — assumption of success — the 
driving mechanism in learning, which is more than the knowledge by the 
students of evaluation criteria, impact factor on academic success is 113%. 
We have to emphasize that in order to achieve this level, students should be 
accustomed with IBSE. Years of permanent, daily IBSE implementation in 
classroom have to be seen as a mandatory prerequisite for this LBB approach. 

e The ability of students for revealing similarities and patterns, which has an 

impact factor equal to 132%, is the ability to connect the new information 
with their previous knowledge “that is not forgotten”. Thus, this approach 
relates to the formation of analytical and critical thinking skills, the basis for 
lifelong learning skills. 

In this way in order to maximize the academic achievement of students, we should 
increase the independence degree of their learning. In other words, teaching should be 
seen as couching in sport when the athlete not only knows what the couch wants from 
him/her, but also assumes these tasks as own goal and has all physical, technical, tactical, 
and emotional means for achieving the goal set initially by the couch. 
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Learning Effort and Metacognition 


The main goal of contemporary education is the scientific understanding of the 
world by the majority of population. Despite many and various efforts for identifying 
adequate strategies for this goal, it remains Fata Morgana in education. The cause is 
minimisation of the importance of the learning effort or the search for royal ways in 
science. Learning through intellectual effort is a well-known didactical principle that 
reflects the fact that the ascension is only uphill. It suits to Vygotsky’s concept of the zone 
of proximal development. Namely because of this ludic education has a negative impact 
factor. Intrinsic intellectual effort is the base for conceptual, scientific understanding of 
phenomena and things. There can be no understanding without student’s intellectual effort 
in the atmosphere of empathy in classroom that encourages verbalisation and fosters 
communication. We should speak about education centred on student’s learning effort 
rather than student centred education. Constructivist approach without learning effort 
is non-sense. Research suggests that constructivist teaching approach is not necessarily 
reflected in higher test scores but students familiar with constructivist classrooms scored 
much higher when they were asked to explain scientific phenomena, than students in 
traditional classrooms (Berube, 2001). Other results of pilot studies suggest that teaching 
programs based on 5E constructivist approach (engage, explore, explain, elaborate, and 
evaluate) increase motivation and remove boredom by creating positive impression to 
the learners (Arioder et al., 2020). Indeed, motivation and students’ interest have to be 
seen as the first bricks in the foundation of any LBB approach when students’ ownership 
of cognitive goals is targeted. 

Learning by being concept is about not only students’ assumption of learning 
process but also requires that students themselves are aware and understand their way of 
thinking in the case of learning, which is more than the application (or replication) of a 
learning activity taken from the teacher. It is about metacognition when students: a) are 
able for and analyse what type of strategy they will use to accomplish a particular task; 
b) estimate their possible result; c) analyse their result and decide if it is necessary to 
change the strategy to accomplish the task (Kirschner, 2006). Thus, the metacognition 
closely relates with the didactical principle of consciousness of learning. 


Student — Teacher Mutual Feedback 


Two-way communication channel is the mandatory component ofany constructivist 
didactic strategy. According to VTL theory, teaching and learning should not be with 
closed eyes — the teacher doesn’t know how the students have received his/her message, 
and students don’t know and don’t understand what the didactic objectives of the lesson 
are (Hattie & Donoghue, 2016). VTL analyses and measures the influence of more than 
250 factors on academic achievement of school students. From these factors, several 
factors that directly influence the feedback have been selected. For example: 

eTeacher’s a priory assumption about the student’s potential and aptitudes — with 

an impact factor equal to 162% 

eTeacher’s instant knowledge on students’ response to his/her didactic action — 

impact factor equals to 129% (in other words teacher knows immediately what 


students think about his/her action). 
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eTeacher’s ability to coordinate classroom debates within and among groups — 
82%. 

ePeer instruction — 74%. 

eFormative evaluation — 48%. 

e Student centred teaching — 36%. 

e Problem — based learning — 26%. 

The last two strategies are below the benchmark level of 40% mentioned above 
because in the case of student-centred teaching we have the student as the passive object 
of teacher’s action and because the majority of teachers don’t apply permanently PBL 
in their classes. These results are confirmed by PISA results in the Republic of Moldova 
where in three — four classes there is only one student with high depth of knowledge, 
compared with South Korea where there are three -four such students in one classroom 
(PISA, 2017). 

The technical solution for instant feedback is digital classroom response system. 
Having such a system, the teacher will not "advance" to the next topic until at least 
70% of students will understand the new notion or law. In this way, the instant teacher’s 
knowledge of the effect of his/her didactic action on the students’ understanding is three 
times more valuable than the assumption that the student would have retained something 
(knowledge of the effect — 129%, assumption — 40% (Nicol & Boyle, 2003). 


Learning by Being within Inquiry-Based Science Education 


Further, we will present the main steps to be followed by the teacher in the 
classroom, in order to achieve the assumption of the cognitive objectives by the 
students. In other words, how to put into practice the LBB concept in the frame of an 
inquiry-based science education (IBSE) project carried out by students. Here we present 
separately the teacher’s role within IBSE lesson (namely teacher — student interaction, 
and students’ group activities. Related to teacher we will analyse three moments: a) 
formation of students’ ideas (this fact determines to what extent the students will assume 
the cognitive goals); b) guidance of students’ own research activity, developed in groups; 
c) facilitation of analysis and drawing of conclusions (within debates at group and 
class level). Related to students’ action we will describe: a) students’ mandatory steps 
within IBSE classroom; b) development of group work; c) recording and analysis of 
measurement and/or research results, etc. 

Teacher — student interaction 
1. Formation of students’ ideas 
In order to reveal students’ initial ideas, the teacher asks open questions and 
gives positive feedback to let students realize the necessity for the revision or 
development of their initial ideas. 
2. Supporting students’ own research 
The teacher helps students to formulate productive questions such as "What is the 
shortest path, convenient form, etc. ...?" - which will form the basis for further 
research and guides the students in the planning of their further research. The 
teacher facilitates the planning process, providing a possible structure of the 
research plan. In addition, the teacher helps the students to record the measured 
data and to organize them in tables. 
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3. Guidance of students within analysis process and formulation of conclusions 
The teacher asks the students to formulate as concisely as possible their 
conclusions, not only to say what the result of the measurements is, but what its 
meaning is. The teacher asks the students to check if the conclusions formulated by 
them correspond to the results of the measurements or observations. The teacher 
encourages students to reflect on what they have done and what conclusion they 
have reached. 

Student activities 

4. Conducting research 

Based on their ideas, students formulate and explain their hypotheses. Students 
participate in planning the experience by discussing the details. Students do not 
create the research plan, but at least they understand and accept it. Here, in fact, we 
are talking about assuming the objectives of learning — according to the concept 
of LBB. Students analyse, compare, and explain the data obtained in relation to 
their hypotheses. 

5. Group work 

Students collaborate and share tasks being in the same group. Avoid individual 
work. Students report the results of group work in front of the whole class. 
Students — rapporteurs answer the asked questions, others approve, reject, or 
debate the results of the given group. 

6. Recording the results 

Students record what they have measured and what they have identified. All 
collected data are organized in the form of a table, i.e., any register must contain 
a table, so that students are familiar with the organization of the data collected in 
a table. Students note in the register if their results correspond to their hypothesis. 
The conclusion reached by the group is written in the register. Students also make 
their own notes, informal, with ideas or data, which they consider appropriate. 
Although informal in their content, these notes should be mandatory in order to 
form lifelong learning skills through inquiry-based learning. 


Conclusions 


This short communication presents the evolution of educational constructivist 
paradigms from the concept of earning by doing to the one of learning by being in terms 
of their impact on academic achievement of school students. The author emphasizes 
that active learning is not yet understanding, and its impact factor could be lower than 
conventional teaching. Moreover, apparent understanding is not equivalent to the 
formation of lifelong learning skills, or to the formation of that knowledge “which is not 
forgotten”. 

In order to achieve its main goal, namely scientific understanding of the world by 
the majority of population, the modern education should be based on students’ learning 
effort and make them assume the ownership on learning goals. More simply, for students 
the desire and effort for knowledge should come intrinsically — the core idea of learning 
by being concept. Students’ intrinsic intellectual effort is the base for their conceptual, 
scientific understanding of the world of things and phenomena. 
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Along with intellectual effort, learning by being is based on metacognition, 
which relates with the didactical principle of consciousness of learning. It means that 
students not only understand what they do but understand their own way of thinking. In 
these given conditions, the teacher turns into a coach who must provide the emotional 
environment for guided self-scaffolding. 

The concept of learning by being changes the format of teacher — student 
communication. For a fruitful learning process, we should have a permanent student — 
teacher mutual feedback. It is according with the theory of visible teaching and learning 
when both the student and the teacher have the same cognitive/didactic goals. In this 
sense, the author emphasizes that feedback — based strategies have the biggest impact 
factors. However, the most valuable strategies are those focused on the management of 
multilateral feedback or teacher’s ability to coordinate the group work within inquiry- 
based classroom activities. 

In order to give practical solutions for application of the concept of learning 
by being, the author structures both teacher and students’ activity in classroom on the 
example of inquiry — based science education project. Thus, in the development of 
fruitful teacher — student interaction the teacher will focus on the following aspects: the 
formation of students’ ideas; supporting students’ research; guidance within analysis and 
drawing conclusions. Related to students, the most important for teacher is to focus on 
making students assume the objectives of the research, to manage their group work, and 
get them familiar with discussions and debates. 
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Abstract 


Education research has repeatedly shown that active learning in physics is pedagogically 
more efficient than traditional lecture courses. Widespread application of the active learning is 
slowed down by the lack of data on the performance of the active learning in widely varying 
circumstances of different educational systems. We measured the level of understanding of basic 
Physics concepts using Force Concept inventory for students who enrol at different universities in 
Latvia in calculus-based and non-calculus-based groups and compared the student performance 
to the pre-test results elsewhere in the world. We measured the growth of concept inventory test 
results and studied the dependence of the growth on the teaching approach used by university 
lecturers. About 450 undergraduate students from 12 groups of science and engineering courses 
taught by 8 lecturers were involved in the study at three universities in Latvia. The Force Concept 
Inventory multiple-choice test was translated to Latvian and used for pre-/post-tests. The pre-test 
results showed that the maximum of the distribution of correct answers for non-calculus groups 
is around 20%, which is the value obtained by the random guessing of test answers, whereas 
the pre-test results of calculus-based groups was about 50% of correct answers. The test score 
after taking post-test confirmed that the growth of students’ tests results is closely related to the 
teaching approach chosen by lecturer, showing that in order to provide physics graduates with a 
good conceptual understanding of physics, student centred teaching approach was crucial. The 
use of concept inventories in undergraduate physics education to measure the progress of learning 
appears to be particularly important in the current situation with a small number of students in 
Physics and a critically small number of future physics teachers, when efficiency of teaching is of 
crucial importance. 

Keywords: STEM education quality, conceptual understanding, student-centred approach 


Introduction 


In Latvia students are enrolled at universities after completing secondary 
education and after participating in Centralized Examinations organized by the State. 
There is no mandatory final exam in physics and students can enrol in the science and 
engineering faculties in Latvia without a physics exam grade. It is possible to be accepted 
at university only by the result of a mathematics final exam. Some Upper-secondary 
school graduates, who see their future in physics or engineering, choose the State 
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Centralized Exam in Physics. Since at the undergraduate level of physics, students may 
enrol at universities without physics exam, the teaching staff at universities must reckon 
with unpredictably varying initial knowledge and understanding of physics for different 
students at the beginning of physics courses. Lecturer’s claim that the understanding of 
physics after upper secondary is generally low, but it is only a subjective point of view, 
usually communicated without further justification or quantitative measurement. The 
teaching of physics courses also benefits from measurement allowing to quantitatively 
characterize the growth of student’s understanding of physics and growth of skills to 
apply physics laws, depending on the learning approach used in a course. 

In order to measure the understanding of student’s basic concepts of physics 
at the beginning of undergraduate physics courses and to determine the growth of 
understanding, Force Concept Inventory (further FCI) was used at several universities of 
Latvia during this research in 2018-2021. 

Research Questions: 

e How high is the level of understanding of the basic physics concepts of 
students who enrol in calculus-based and non-calculus-based groups, as 
objectively assessed by concept inventory and how does this level compare 
against FCI pre-test results elsewhere in the world? 

e How highis the growth in concept inventory test results for students depending 
on teaching approach used by university lecturers? 


FCI was developed and first applied by Hestenes, Wells and Swackhamer (1992, 
March). The evidence of their study was essential: “the knowledge gained under 
conventional instruction is essentially independent of the teacher” (Savinainen, 2002). 
A revised version of the FCI was developed and placed on the web in 1995 (Halloun et 
al., 1995) and later appeared in Mazur’s book (Mazur, 1997). A detailed analysis of FCI 
results with large statistics, including six thousand students in the study, was performed 
by Hake (1998). 

The growth of student understanding of basic physics concepts is measured as 
FCI gain. Gain described in studies ranges from about 20%, which is just a fifth of 
the student's potential achievement, up to about 70%. For example, Georgia States 
Universities, US, recent study using FCI test included more than 5,000 students showed 
the average gain that range from 47.4% up to 71.3% (Caballero, Greco, Murray, Bujak 
& Marr, 2012). Similar results are seen in two other studies: at the University of Toronto 
first year Physics course (Lasry, Watkins, Mazur & Ibrahim, 2013) reports gain values 
of 34.03-45.02%); in the Physics and Astronomy New Faculty Workshop (Lee, Manju, 
Dancy, Henderson & Christensen, 2018) the average gain between 40-60% was observed. 

So far, the FCI has been administered to countless thousands of students at many 
universities worldwide. The FCI has not lost its relevance and has been re-examined 
and as the research by Von Korff et al. (2016) suggests, interactive engagement teaching 
techniques are significantly more likely to produce high student learning gains than 
traditional lecturing approach. 
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Research Methodology 


Background 


The study was initiated at University of Latvia. It relies on FCI test as ameasurement 
tool. FCI test consists of two parts: pre-test and post-test composed of multichoice 
questions and is given at the beginning and end of the course. The measurement is 
performed by calculating the absolute and relative difference of the results (Hestenes et 
al., 1992; Lasry et al., 2013). In order to use it at universities of Latvia the FCI test was 
translated into Latvian and was given to students by several physics’ lecturers during the 
period 2016-2018. During this period experience with the use of the test was gained and 
language translation inaccuracies were eliminated. 

In the first year of the study, FCI pre-test and FCI post-test were carried out at 
University of Latvia only for one first year undergraduate calculus-based physics course 
in 2018, and the results were reported at the BalticSTE'19 conference by Cinite and 
Barinovs (2019, June). After that, the research was continued for students at several 
universities until autumn 2020, when due to remote training it was decided to not use 
the FCI test remotely. At this time, the test was administered, and data was analysed for 
12 undergraduate student groups at three universities in Latvia involving approximately 
450 students in total. 


Participants 


Students of undergraduate physics courses of three universities: University of 

Latvia, Riga Technical University and Ventspils University of Applied Science, were 
involved in this research during the time period 2018-2020. The research includes 445 
student's pre-test results. 
Not all students performed a post-test, and there were a number of reasons: firstly, for all 
groups studies started in person, but three out of twelve groups were forced to continue 
the studies remotely soon after the course started due to the Covid-19 restrictions; and 
several students did not complete the post-test due to other reasons. The post-test could 
not be performed if the physical presence of students were not possible during post-test. 
344 FCI post-test results were obtained, but only 342 students participated both in pre- 
and post-test and their results were used to calculate the relative and absolute test score 
gain in nine groups. 

About a quarter of students or 111 students were in calculus-based groups. 
Participants were approximately 18—20 years old, slightly more than half or 58% of all 
students were male and 42% female. The classes were taught by 8 lecturers at different 
levels of the academic ranking: three associate professors, three docents and two 
lecturers. Three of the lecturers have a doctor (Dr.) degree and other five have a masters 
(Mg.) degree. 


Procedure 


The FCI test has been used in a research by the University of Latvia since 2018. 
Two years before the start of this research, the FCI test was translated into Latvian 
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and approbated to several groups of students and checked by university teaching staff 
to prevent translation errors. The original FCI procedure was retained in all research 
groups: the pre-test was given to students in the first lecture, before starting the course, 
the post-test was given out at the end of the relevant topic of the mechanics or at the end 
of a course of mechanics. Both parts of the test were carried out by students in-person, 
in-classroom, in writing. When it was possible, the course lecturer did not know the test 
questions in order to prevent the lecturer from preparing the students to correctly answer 
the specific answers of the post-test. 

As in our previous study (Cinite & Barinovs, 2019) teaching effectiveness is 
characterized by relative learning gain, (g), which is defined as the relative change of test 
results with respect to the maximal possible test result increase from pre-test to post-test 

( ) __ (Post)—(Pre) 
Gg) ~ To0- (Pre) ’ 


where (Post) is group average post-test result and (Pre) is group average pre-test result. 
The relative gain is used since it less depends on the initial knowledge on students and 
allows to compare groups with different pre-test results. Our calculated gain can be 
compared to the 0 to 34% average gain (g) that has been observed for traditional teaching 
approach and classes with low student engagement (Hestenes et al., 1992; Lasry et al., 
2013). 


Research Results 
FCI Pre-test Results 


Data for all 12 group pre-test results are collected in the Table 1. The group’s 
average pre-test results are in the 18-54% interval of correct answers. Pre-test results 
for three calculus-based groups are relatively high, in the interval 49-54%; for nine non- 
calculus-based groups the number of correct answers is between 18% and 37%. The FCI 
post-test was administered only for nine groups, as three groups started to study remotely 
after the pre-test. The average relative gain and the average absolute gain of each group 
was determined for nine groups possessing post-test results. 
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Table 1 
FCI Test Results for Twelve Student Groups, 2018-2020 





Group av- Group Group Learning 
erage FCI average average approach 
post-test absolute relative © S— student 


Group average 
N. Semester FCI pre-test 









































eu result gain gain centred, 
L —lecture 
% % % % centred 
1 Fall 2018 54 65 11 27 L 
23 Z 2 Faiz 49 81 32 65 s 
S88 & 3 Fall2020 51 continued learning remotely 
4 Fall 2018 37 49 12 23 L 
5 Spring 2019 22 48 26 33 $ 
B 6 Fall 2019 23 54 31 40 $ 
3 7 Fall 2019 25 60 35 47 $ 
3 8 Fall 2019 26 43 17 25 L 
8 9 Fall2019 23 61 37 51 s 
3 10 Spring 2020 26 continued learning remotely 
8 11 Spring 2020 18 continued learning remotely 
S 42 Fall2020 22 48 26 34 s 
FCI Gain Results 


We separately analysed results for two groups: the results of groups where 
lecturers worked with a lecturer-centred approach and the results of groups where 
lecturers worked with a student-centred approach. In the student-centred classes 70— 
80% of time was spent on lectures and 20-30% of time was spent on peer instruction 
(Mazur, 1997). Lecturer-centred approach classes consisted of information presentation 
by lecturers followed by solving standard end-of-chapter exercises at different levels of 
difficulty. Several lecturers in student-centred classes were considerably less experienced 
than lecturers in lecturer centred classes. 

In each of these groups, we analysed the results of calculus-based groups and the results 
of non-calculus-bases groups separately. 

For a calculus-based group with a lecturer-centred learning approach, the group's 
relative gain is 27%, for a group with a student-centred learning approach average 
relative gain is 65% which is the highest result in this study. Similar difference of gain in 
student-centred classes being higher is for non-calculus-based groups. In these groups, 
a lecturer-centred approach for two groups has resulted in relative gain 23% and 25%, a 
student-centred learning approach for five groups has yielded 33-51% range for relative 
gain. 

The results are shown in Figure 1, where the idea and visual style of the Figure is 
adapted from Hake’s study (1998). The Figure 1, where different symbols: a blue triangle 
and a yellow diamond, highlights separately the groups in which the lecturers worked 
with different learning approaches. 
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Figure 1 
Pre-test Results versus Absolute Gain 
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Three different colour bars in Figure 1 show the level of the relative gain 
achievement: ‘Low-g’ result of the groups , a ‘Medium-g’ result , a “High-g’ result . 

As shown in Figure 1, for those groups where lecturers worked with a student- 
centred approach, the results are found in the medium-gain area. Regardless of the pre- 
test results, students were able to achieve a relatively high gain. For lecturer-centred 
learning approach groups, regardless of the outcome of the pre-test, the gain is placed 
in a low-achievement area. Among these groups, there is also one calculus-based group 
with the highest pre-test score of 54%, but only 27% relative gain result. 


Normal Distribution and Random Probability of Results 


The maximum for probability distribution of correct answers in the pre-test for 
non-calculus groups is close to 20%, which 1s close to expected maximum for the random 
guessing of the test answers in a multiple-choice test with 5 answers for a question. 
In order to see what part of correct answers could be explained by a simple chance, 
we have compared the results of the pre- and post-tests with a distribution for random 
guessing one correct answer out of the five equally possible answers. As can be seen 
in Figure 2, the pre-test results of non-calculus-based groups with N = 335 students 
remarkably well coincide with the binomial distribution generated by random guessing 
and both distributions are almost indistinguishable. That suggests that students in non- 
calculus groups might not possess the knowledge of the basic concepts of physics prior 
to enrolment at university even if their score is above the average and their higher score 
might be by a pure chance. On the left side of Figure 2, the centre of distribution of 
correct answers for students in calculus-based groups is shifted to the right comparing 
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to the probability distribution predicted by random guessing and points to some initial 
knowledge of students. 


Figure 2 
Histogram of the FCI Pre-test Results 
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| 
Note: A — non-calculus-based and B — calculus-based students; and dashed line of random probability 
distribution to answer correctly one of the five possible answers for each question. 


In the post-test results for all groups the random probability distribution is placed 
to the left from distribution generated by students’ answers, (see Figure 3) showing the 
growth of student knowledge in both groups. 


Figure 3 
Non-calculus Groups FCI Post-test Results 
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Note: A— Lecture centred groups and B — Student centred groups result. 
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Discussion 


The results of this study show that the groups in which lecturers worked with a 
student-centred approach were able to achieve high relative Gain above 30% despite 
the low pre-test results for the students. This agrees with the original study of the FCI 
(Hestenes et al., 1992) that found high relative Gain above 34% achieved by lecturers 
working with a student-centred approach. Groups, where lecturers worked with a 
lecturer-centred approach, demonstrated low efficiency, irrespectively to the group's pre- 
test results. The gain results of this research demonstrate that in universities in Latvia 
relative gain falls in intervals of 25-65% similar to the studies elsewhere in the world. 

The pre-test results for non-calculus-based groups showed that the maximum of 
the distribution of correct answers is around 20% which is the value obtained by the 
random guessing of one correct answer of five multichoice test answers. The pre-tests 
result of calculus-based groups was about 50%. The post-test result confirmed that the 
growth of students’ tests results is closely related to the teaching approach. 

In this study the gain as measured by FCI did not depend on lecturer’s scientific 
degree or lecturer’s status on an academic pedagogical or academic researcher level. 
This also has been observed in the study of Hestenes and Halloun (1995). 


Conclusions and Implications 


The results of the 4-year study on FCI pre-test for 12 undergraduate level student 
groups have been compiled from 445 students who learned physics in science and 
engineering courses of three universities in Latvia. To our knowledge, this is the first 
published FCI study in Latvia and the Baltic states. 

FCI pre-test results show the level of students' understanding of the physics basic 
concepts at the beginning of the studies: the mean pre-test scores for the three calculus- 
based groups ranged from 49% to 54%, and for the nine non-calculus-based groups 
ranged from 18% to 37%. Relatively high pre-test scores for calculus-based groups may 
be explained by students' motivation to study physics, this and other variables should be 
included in a future research. The distribution of the pre-test results of the non-calculus- 
based groups is close to the random probability distribution when students correctly 
guess one of the five possible answers to a question. 

The achievement of the student groups as characterized by the relative gain of 
pre-/post-test results correlates with the lecturer's teaching approach. Groups where 
training took place through a lecturer-centred approach reached the Relative Gain in 
the range of 23-27%; two of those groups were non-calculus-based, one calculus-based 
group. 

The result of about 25% reached in lecture-centred learning groups shows that 
students had been able to reach just a quarter of possible gain on a basic mechanics 
comprehension test. Those groups also had relatively more students that showed a 
negative gain. This could indicate a student's confusion in understanding of the basic 
principles of physics and following guessing of correct answers in the post-test. 

Groups where lecturers worked with a student-centred learning approach have 
shown larger growth in conceptual understanding with the relative gain above 34%. The 
highest result 65% was observed for the student-centred working calculus-based group. 
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The main bulk of studies using FCI is done in United States. This research 
allowed to verify the FCI in the context of very different education system of Latvia. The 
research confirmed that in order to provide physics graduates with a good conceptual 
understanding of physics as measured by FCI, student centred approach is crucial. 
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Abstract 


This review study includes 19 articles from 2016 to 2021 focusing on in-service science teachers’ 
professional development targeted to promote student conceptual understanding. The present study 
is guided by the following research question: “What characterizes high-quality in-service science 
teachers’ professional development targeted to promote student conceptual understanding?” 
The review indicates that such classroom practices as modelling, questioning, and arguing from 
evidence are perspective ways to develop student conceptual understanding in science classrooms. 
A mixture of input, application, and reflection; long-term involvement of participants; focus on the 
question how to foster transfer from teacher professional development into participants ’ everyday 
work characterize high quality teacher professional development interventions that develop and 
support inquiry practices. Results suggest that teacher professional development that is sensitive 
to teacher learning needs is a way to develop student conceptual understanding. 

Keywords: in-service teacher professional development, science teacher education, student 
conceptual understanding, teacher learning 


Introduction 


In the last decades science curricula reforms worldwide have emphasized the 
importance of fostering pupils' scientific literacy and developing students conceptual 
understanding through authentic and relevant scientific practices (European 
Commission., 2011; Eurydice, 2011; Next Generation Science Standards: For States, by 
States., 2013; OECD, 2006). Concepts and principles are the basic building blocks of 
scientific knowledge and understanding of a concept is precondition for making complex 
inferences or accomplishing any scientific work with it (Mi et al., 2020). Therefore, it is 
decisive for students to attain conceptual understanding about the core ideas of science 
subjects and build this understanding coherently from K-12. 

Teachers and their classroom practice are considered as primal conciliators for the 
transmittance of reform efforts into student learning outcomes. Therefore, it is critically 
important to support the development of teachers for the challenge of the reforms (Yang et 
al., 2018). According to Desimone's (Desimone, 2009) model of effectiveness of teacher 
professional development (TPD), TPD may increase teacher knowledge and change 
teacher attitudes and beliefs toward teaching, thereby improving teacher classroom 
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practice, which in turn results in increased student learning and achievement. This model 
has been widely accepted, and adopted, in science education. Therefore, high-quality 
TPD programs are needed to support the development of teachers to adequately align 
their teaching with the reforms (Yang and Liu, 2020). 

Latvia, like other countries, is undergoing a curriculum reform (Nieveen, 2018) 
that has a focus on 21st century skills, complex use of knowledge, skills, attitudes, 
and values. In Science subjects and Math, the reformed curriculum includes new goals 
linked to deep understanding of subject-specific and crosscutting scientific concepts. 
Thus, it becomes increasingly important to monitor if and how teachers plan and ensure 
the classroom learning and whether it aligns with the goals of the reform. This then 
provides information for planning and delivering the necessary TPD support. Still 
TPD opportunities for in-service science teachers, that are focused to promote student 
conceptual understanding, are missing (Namsone, 2018). The local context of Latvian 
science teacher professional development led authors to the current review of recent 
research focusing on in-service science teachers’ professional development targeted to 
promote student conceptual understanding. Both authors are taking part in the planning, 
conduction, and evaluation of TPD, so the following research question from TPD 
organiser point of view arouse: “How the criteria of high-quality TPD manifest into in- 
service science teacher professional development targeted to promote student conceptual 
understanding?” The research was aimed to describe the most recent research findings 
focusing on in-service science teachers’ professional development targeted to promote 
student conceptual understanding. 


Research Methodology 
Search Criteria 


To answer the research question, we conducted a search about pedagogy in the 
ISI WEB of Science (search undertaken |. April 2021) using the search strings “teacher 
professional development” and “student conceptual understanding.” The search focused 
on the years from 2016 to 2021 to cover the most recent research. Intending to obtain 
an overview of previous research published in international research journals related to 
teachers’ professional development, the authors obtained 73 hits for all the search strings 
together after narrowing the search down to “education and educational research.” 
Articles were included if they dealt with basic education in primary and secondary 
school. Exclusion criteria included reviews, articles that dealt with pre-service teachers, 
special education, higher education, pre-school education, informal learning, and 
teachers’ individual learning. Each of these areas certainly would benefit from separate 
review studies. In this way, the present study focused on experienced basic education in- 
service science teacher professional development. Based on the abstracts of the identified 
articles, we selected a set of 59 articles for thorough reading. After reading all 59 articles, 
a final set of 19 articles were chosen on the basis of the same exclusion and selection 
criteria listed above. Altogether, 2 of these articles followed qualitative methods, six 
employed quantitative research, and eleven involved a mixed-method approach. The 
published studies mostly came from the USA (15 articles). Single studies came from 
South Africa, Canada, Saudi Arabia, and Singapore. The selected papers offered both 
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breadth and depth, offering insight into the research question of what characterizes 
high-quality teachers’ professional development targeted to promote student conceptual 
understanding. 


Selection of Review Categories 


The framework describing high quality PD interventions developed by European 
STEM Professional Development Centre Network (European Network of STEM PD 
centres, 2019) was used to examine the articles from a TPD-developer point of view to 
identify, how the criteria of high-quality TPD manifest into various TPD interventions. 
According to the framework, high-quality science TPD interventions: 

1) involve participants in forms of active learning, 

2) provide long-term involvement of participants, 

3) are sensitive to and consider participants’ learning needs, 

4) consider and foster teachers’ scientific competence, 

5) address knowledge, skills, and attitudes relevant to the curricula, 

6) foster the transfer from TPD into participants’ everyday work, 

7) evaluate the impacts of the TPD. 

We structured and compressed the articles by coding and categorizing the texts in 
selective, open, and axial analysis processes (Strauss and Corbin, 1990, 1998), rendering 
their essence reportable (Garfinkel, 1967; Sachs, 1992). This selective analysis process 
enabled selection of a core category; in this study, the core category had been chosen 
according to the criteria number 5: high quality in-service science teachers’ professional 
development targeted to promote student conceptual understanding. 

To define the sub-categories for this review, the criteria number | to 4, 6 and 7 of 
high-quality science TPD interventions were presumed as variables that can be identified 
in the studies describing TPD interventions and change according to different contexts 
and goals. 

According to the chosen criteria of high-quality science TPD six sub-categories on 
the same horizontal level were defined: (1) how to involve participants in active learning, 
(2) how is long-term involvement of participants ensured, (3) how are participants’ 
learning needs considered, (4) how is transfer from TPD into participants classroom 
practice fostered, (5) how is the development of teacher scientific competence fostered, 
(6) how are the impacts of the TPD measured. 


Research Results 
Participant Active Learning 


Most of the studies reviewed focused on active participant involvement where 
teacher-learners engage in inquiry-based input activities followed by reflection about the 
experience and modelling the application of the learned content. Yang and Liu (2020) 
highlighted such approach. An input workshop in their TPD program consisted of six 
parts: (1) a short review, or lesson sharing; (2) a discussion of the focus of the workshop; 
(3) a first group activity or discussion based on the focus; (4) a second group activity or 
discussion based on the focus; (5) a presentation by the research team or guest speaker; 
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and (6) a debriefing and session evaluation. Not only in the TPD program developed by 
Yang and Liu (2020), but also in programs developed by Shernoff et al. (2017), Miller 
and Kastens (2018), Chen et al. (2017), Wilhelm et al. (2018), Drewes et al. (2018) 
and Murphy et al. (2018) such input workshops were used in pre-schoolyear summer 
institutes and in-year follow-up sessions. TPD models that combine summer and/or 
winter institutes with monthly follow-up sessions and independent implementation of 
learned into practice between the follow-up sessions have shown satisfaction between 
teacher-participants (Shernoff et al., 2017) and positively impacted student conceptual 
understanding (Miller & Kastens, 2018; Wilhelm et al., 2018). Binmohsen and Abrahams 
(2020) and Yoon et al. (2020) have investigated how programs developed according to 
similar TPD model have been conducted online. Their findings suggest that online TPD 
can be as effective as, and in certain places more effective than, the face-to-face TPD. 

Yoon et al. (2020) distinguished social ties, which can be developed through 
discussion boards or experience and resource sharing, as a key aspect of online TPD. 
Another approach that enables active involvement of participants in TPD is collaborative 
action research (CAR). Both Yin and Buck (2019) and Goodnough and Goodnough 
(2018) investigated how CAR can be used in TPD that is focused on implementation 
of classroom practices that promote student conceptual understanding. Both groups of 
authors concluded that CAR approach in TPD requires additional support to the teacher- 
learners, who can border with individual mentoring, especially when novice teachers 
take part in CAR. 


Long-term Involvement of Participants 


11 of 19 studies reviewed included TPD interventions that lasted for more than 
1 year. TPD interventions carried out by Yang and Liu (2020) and Chen et al. (2017) 
can be mentioned as the most longitudinal, lasting 5 and 4 years respectively. In both 
programs the long-term participant involvement was ensured through the summer 
institute — regular follow-up session model. Through participant questionnaire Yang 
and Liu (2020) identified an interconnection between teacher summer placement in 
research facilities and teacher-participant long-term involvement in TPD. Participant 
questionnaire answers indicated that the authentic experiences of scientific practice that 
teacher-learners experienced served as motivators to implement and teach the elements 
of these practices in their science lessons. 


Participant Learning Needs 


Only few of the reviewed studies describe the identification and consideration 
of participant learning needs. An example of such practice was described by Miller 
and Kastens (2018). The authors investigated how implementation of dynamic 
physical models in classroom practice impact student conceptual understanding. The 
TPD included in the research started with teacher-participant interviews where in the 
curricula they teach they see the right fit for extended modelling practice. The further 
TPD program and implementation in classroom practice focused on learning units 
identified by teachers. The authors reported active teacher involvement in the two years 
of the TPD program and increase in student conceptual understanding regarded to the 
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curricular topics, where teachers implemented extended use of dynamic physical models. 
Another example of the evaluation of teachers learning needs was described by Stott 
(2020). Before a TPD intervention to promote conceptual understanding about chemical 
equilibrium, the author carried out a belief questionnaire and a biographical survey. Data 
analysis led to the identification of teacher sub-groups and development of two different 
TPD interventions. One — to foster the keystone practices to promote student conceptual 
understanding, the other — to focus on more complex classroom practices that promote 
student conceptual understanding. 

The discussion section of almost all the articles reviewed mentioned participant 
needs as an area for future research. Logical connections between consideration of 
participant needs and mixture of input, application, and reflection; long-term involvement 
of participants and transfer from PD into participants’ everyday work can be identified 
(Figure 1). 


Figure 1 
The Identified Connections between Criteria of High-quality Science TPD 
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The scheme, presented in Figure 1, doesn’t include the criterion that high-quality 
science TPD interventions must foster teacher’s scientific competence. Learning science, 
the understanding of concepts can’t be isolated from scientific skills. Understanding of 
concepts can be gained and assessed through use of science skills. The combination 
of understanding with skills leads to a complex outcome which is a step in developing 
scientific competence. 


Transfer from TPD into Participants Classroom Practice 
Various ideas how to foster the transfer of the ideas, knowledge and pedagogical 


practices obtained in the TPD interventions can be highlighted in the reviewed articles. 
The studies conducted by Murphy et al. (2018), Yoon et al. (2020) and Binmohsen 
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and Abrahams (2020) described ready-to-use classroom resources as mediators for the 
implementation of classroom practices that promote student conceptual understanding 
into everyday work. In contrast Shernoff et al. (2017) used writing of curricular unit 
plans and lesson plans as a method to foster the transfer from TPD into participant’s 
everyday work. The authors report various outcomes as some _ teacher-learners 
struggled with appropriate implication of classroom practices that promote conceptual 
understanding in lesson and unit plans, but some — with implementation of aligned 
assessment strategies. Two case studies conducted by Longhurst et al. (2017 & 2021) 
investigated the mediators that help teachers transfer TPD into everyday work. In both 
studies the authors studied sub-samples of the teachers who managed this transfer 
best. The data collected in this investigation indicated that the personal construction 
and modification by teachers participating in TPD may be critical in achieving high 
levels of appropriation of instructional pedagogies provided during TPD interventions. 
Factors that influence appropriation of professional development and inquiry strategies 
include opportunities for personalized paths to appropriate conceptual or practical tools, 
planning for intentional adaptation allowing teachers to develop ownership attributes 
of pedagogical tools of instruction, and the support or creation of trusting cultures of 
learning where teachers develop long-term support systems. Miller and Kastens (2018) 
came to similar conclusions in their study about implementation of modelling practice 
to promote student conceptual understanding. The authors mention the deep engagement 
with modelling practice within the context of participant subject matter; adequate time 
and support to generate lessons that integrated the proposed strategies while maintaining 
each teacher’s autonomy; TPD interventions built around models that they were already 
using as key strengths of the TPD program. 


Development of Teacher Scientific Competence 


Conceptual understanding is interrelated with skills and beliefs that form scientific 
competence. Such scientific skills as argumentation (Cavlazoglu and Stuessy, 2017; Chen 
et al., 2017; Fishman et al., 2017; Murphy et al., 2018; Osborne et al., 2019), modelling 
(Miller and Kastens, 2018; Wilhelm et al., 2018) can be used as classroom practices 
to acquire conceptual understanding. Conceptual understanding is vital to form beliefs 
who can later form into behaviours and actions that help to solve complex global socio- 
scientific issues (i.e., climate change) (Drewes et al., 2018). Also, student behaviour 
and actions they take to solve socio-scientific issues can be used in tandem with other 
practices to assess student conceptual understanding. 


Evaluation of TPD Impacts 


Most of the studies reviewed use common methods to measure how TPD impacts 
participant practice. To uncover how the TPD impacted teachers practice mixed-method 
approach is common in the reviewed studies. Most of the studies include different 
combinations of such well-known TPD evaluation methods as semi-structured interviews 
(Mulvey and Bell, 2017; Yin and Buck, 2019), lesson observations (Daane et al., 2018; 
Drewes et al., 2018; Yin and Buck, 2019), participant and student questionnaires 
(Binmohsen & Abrahams, 2020; Yang & Liu, 2020), student conceptual understanding 
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tests (Drewes et al., 2018; Murphy et al., 2018; Yang & Liu, 2020), expert observations 
(Daane et al., 2018), and participant lesson plan analysis. Still there were few examples 
of novel methods of TPD impact evaluation. Cavlazoglu and Stuessy (2017) investigated 
changes in science teachers’ conceptions about earthquake engineering. One of the 
methods authors used to determine the changes in teacher-participant macro- and micro- 
level knowledge after TPD intervention was teacher concept map analysis. Sherwood 
and Sherwood (2020) examined shifts in teacher perceptions of classroom instruction 
that promotes conceptual understanding before and after TPD experience. Visual 
research methods were used to collect qualitative data (teacher drawings), which were 
analysed and interpreted using quantitative methods, and compared with content analysis 
of teachers’ written text. 


Conclusions 


The analysis of the articles reviewed showed that almost all TPD interventions 
were designed according to criteria of high-quality science TPD interventions (European 
Network of STEM PD centres, 2019). A TPD model that provides interplay of input, 
application and reflection was summer institute followed by monthly follow up sessions 
during the school year with time for practice and classroom implementation or lesson 
studies between them. Another TPD model that can provide this complex interplay is 
collaborative action research. 

Both TPD models lead to a logical connection between input, application and 
reflection and long-term participant involvement as full implementation of TPD 
interventions developed according to the models takes at least one school year. Both 
models also include independent teacher practice followed by reflection which is a 
way how to foster the transfer from TPD to participant’s everyday work. Personalized 
learning paths, planning for intentional adaptation and the support or creation of trusting 
cultures of learning are some mediators that can develop teacher-learners ownership 
of the learned contents. The evaluation of TPD impacts must include mixed-methods 
approach, combining both qualitative and quantitative methods to evaluate how TPD 
has impacted teacher knowledge, perceptions, classroom practice and student outcomes. 
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Abstract 


Mathematical calculations are an important part of chemistry. Those problems are difficult for 
students, especially if the task is set with a limiting reactant. The aim of this study was development 
of a Procedure for evaluation of cognitive complexity of the Stoichiometric Tasks with a Limiting 
Reactant. The procedure created included an assessment of the difficulty of concepts and an 
assessment of their interactivity. As a research instrument for assessing performance, the test 
of knowledge was specifically constructed for this research. Each task in the test was followed 
by a seven-point Likert scale for the evaluation of the invested mental effort. The research 
included 58 upper-secondary students. The validity of the procedure was confirmed by a series of 
regression analyses where statistically significant correlation coefficients are obtained among the 
examined variables: students’ achievement and invested mental effort from cognitive complexity 
(independent variable). 

Keywords: chemistry education, stoichiometry, problem tasks, achievement, mental effort 


Introduction 


Stoichiometry is one of the key areas of chemistry. The numerical parameters of 
a chemical reaction, the concentration of reactants, the amount of substance, as well 
as the basic parameters of chemical kinetics and equilibrium are calculated based on 
stoichiometric relations (Hanson, 2016). In a chemical reaction, two or more reactants 
react in constant molar relation - they react stoichiometrically. For example, hydrogen 
molecules react with oxygen molecules in a reaction where water molecules are formed 
as the product in a stoichiometric molar ratio of 2: 1. One of the fundamental concepts in 
stoichiometry is the calculation of the quantity, volume, number of particles or the mass 
of the products of a chemical reaction between two reactants where one of the reactants 
is a limiting reactant (Sostarecz & Sostarecz, 2012). 
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In chemical reactions - a limiting reactant, i.e., a limiting reagent, is a reactant 
that first reacts all in a chemical reaction, i.e., it is "consumed" first (Olmsted, 1999). A 
limiting reactant stops a chemical reaction when it fully reacts and therefore determines 
the amount of a formed reaction product (Gonzalez-Sanchez et al., 2014). The tasks with 
limiting reactants are one of the most difficult tasks in stoichiometry (Gulacar, et al., 
2013). Students often use algorithms when they solve problems with limiting reactants. 
They often calculate the amount of products for both reactants, and then they choose 
the one that produces fewer reaction products as the limiting reactant (Toth, 1999). 
Students do not understand the definition of a limiting reactant, what it represents, and 
how they decide on it. Students define a limiting reactant as the reactant with the smallest 
number of atoms (Marais & Combrinck, 2009). By a term limiting reactant, they mean 
the reactant that needs to be quantitatively higher to react completely with another 
reactant (Kashmar, 1997), 1.e., the limiting reactant is the one that has fewer moles in 
the initial system (Gonzalez-Sanchez et al., 2014). In a study by Dahsah and Kol (2007), 
students consider the reactant expressed in the smallest unit of mass the limiting reactant, 
without taking into account the amount of substance and the molar ratio. In the same 
research, some students claim that if both reactants are not present in a stoichiometric 
amount, there is already a limiting reactant, the chemical reaction will not happen, and 
no products will be formed. 

The relation between the reactants which is not |: 1 is confusing for students, 
even when the reactants are expressed in moles (Kalantar, 1985; Hanson, 2016). Similar 
results were found (Dahsah & Coll, 2007), where students always took a mole ratio 1:1 
between the reactants regardless of the coefficients in a chemical reaction equation. In 
a study by Olmsted (1999), this result was previously confirmed in a problem with a 
limiting reactant where the ratio of coefficients was 3:2. Only 30% of students solved the 
task successfully although the reactants were given as the amounts of substances. The 
errors found in his research were that students did not take into account stoichiometric 
coefficients, and they used the wrong algorithm for calculating the limiting reactant. 
Haidar (1997) and Hanson (2016) have even concluded that students take molar ratios 
into account when they calculate substances if substances are given in terms of mass 
units. Hanson (2016) also noticed that there were a misunderstanding and a misuse of 
terms for mass, quantity, and molar mass in calculations in the problem-solving tasks 
with limiting reactants. Coefficients and indexes in chemical formulas are confusing for 
students in a chemical equation, and therefore a common cause of errors in calculations 
is that students misuse coefficients and indexes in chemical formulas (Chandrasegaran 
et al., 2009; Mulford & Robinson, 2002). Thus, Huddle and Pillay (1996) observed that 
a limiting reactant from the students’ perspective was the reactant with the smallest 
stoichiometric coefficient. In research by BouJaoude and Barakat (2000), students chose 
a limiting reactant arbitrarily without a logical explanation, although they usually opted 
for the reactant expressed in units for the amount of substance. In their next research, 
students quoted the fact that it was easier to calculate using moles as the main reason why 
they chose the reactant expressed in moles (BouJaoude & Barakat, 2003). In problems, 
students choose a limiting reactant by observing which of the reactants contains more 
atoms of the elements of the desired reaction product (Wood & Breyfogle, 2006). 

In addition to all the misconceptions, students’ difficulties arising during the 
resolution of limiting reactant stoichiometry problems are caused by the complexity of 
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the tasks themselves. Namely, problem-solving tasks contain a large number of chemical 
concepts that are interconnected by different relationships, which imposes a cognitive 
load on students (Kalyuga, 2009). 

While being processed in the working memory, new information is recombined 
and brought into connection with already acquired knowledge and then stored as 
schemas in the long-term memory. The working memory becomes overloaded if it 
simultaneously processes more information. That is a cognitive load (Kalyuga, 2009). 
The cognitive load is a multidimensional construct consisting of three measurable 
components: a mental workload, mental effort, and performance. The mental load is 
imposed by the teaching parameters, the mental effort is determined by the available 
capacity of the working memory assigned to solve a task, and the performance is the 
student achievement. According to Sweller et al., (2011), the self-assessment of mental 
effort is the most sensitive component in assessment of the differences in the cognitive 
load imposed by different teacher’s instructions. One of the cognitive load indicators 
which have been used as an objective measure recently is cognitive complexity (Harris 
et al., 2013; Raker et al., 2013). 

Bieri (1955) introduced the term cognitive complexity that reflects a high degree 
of differentiation of the system of the constructs that individuals use. Concerning 
the cognitive complexity of a task, the key component is its characteristics. These 
characteristics can be dimensioned by the experts’ estimation. They can evaluate the 
requirements of the task from the aspect of the cognitive load theory. Recent studies 
have shown that cognitive complexity can be used to predict student achievement in 
assignments and invested mental effort (Knaus et al., 2011; Raker et al., 2013). 

Different tools have been developed - Rubrics for estimating the cognitive 
complexity of problems in chemistry (Knaus et al., 2011; Raker et al., 2013). The 
developed Rubrics were created based on the theory of complexity (Goldreich, 2008), 
which defines a system of more related concepts, and the theory of cognitive load (Sweller 
et al., 2011), which defines interactivity among the concepts in a task. The use of the 
Rubric provides an easy way to quantify the cognitive demands of a problem (Knaus et 
al., 2011; Raker et al., 2013). To assess the cognitive complexity of tasks, experts assess 
the number of elements, i.e., the concepts needed to solve a test task. They also assess 
the difficulty of each concept from students’ perspective as easy, medium, and difficult. 
When all the concepts needed for solving the task are recorded, the complexity of the 
task is determined with the use of the Rubric. After determining the difficulty rating of 
the concept, interactivity is calculated. Interactivity increases the cognitive complexity 
of the task when students need to use the interdependence of the components to solve the 
task. Interactivity is assessed as insignificant, basic, and complex. Further development 
of the Rubrics for the cognitive complexity rating is presented in the works of Horvat et 
al. (2016, 2017, 2020). 


Research Problem 


So far, the Rubrics for the cognitive complexity rating in several domains of 
chemistry have been developed. All of them show satisfactory coefficients of the 
correlation between cognitive complexity and student performance, and between 
cognitive complexity and mental effort. However, there was a need for the correction 
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due to the specificity of domains in all the Rubrics. One of the many concepts present in 
stoichiometry problems that further complicate them is the concept of a limiting reactant. 
Problem-solving tasks with a limiting reactant are characterized by high complexity, and 
therefore they are often confusing for students. In this paper, a Procedure for assessment 
of the cognitive complexity of stoichiometric tasks with a limiting reactant has been 
developed. 


Research Focus 


This research aim was to develop and validate the procedure for determining the 
cognitive complexity rating of the stoichiometric tasks with a limiting reactant. The 
specific research objectives for determining a numerical rating of cognitive complexity 
were: 

e Construction of a Table for assessing the difficulty of concepts and their 
interactivity needed for the assessment of cognitive complexity of chemical 
technology problem-solving tasks; 

e Determination of cognitive complexity of test tasks using a combination of 
a constructed Table for assessing the difficulty of concepts with a cognitive 
complexity rating rubric proposed by Knaus et al. (2011); 

From specific research objectives research questions for validation of the procedure 

were created as follows: 

* Is there a statistically significant correlation relationship between the students’ 
performances and the numerical rating of the cognitive complexity of 
problems, and 

¢ Is there a statistically significant correlation relationship between the invested 
mental effort and the cognitive complexity. 


Research Methodology 
General Background 


The research was conducted in May 2017, during the second semester of the 
school year 2016/2017. Students solved the test of knowledge with 7 tasks. With the 
test of knowledge, a collection of students’ achievements and a collection of students’ 
invested mental effort were collected. Students’ achievements and invested mental effort 
were dependent variables, and previously determined cognitive complexity was an 
independent variable. Validation of procedure and research instrument was confirmed 
with basic statistics parameters, descriptive statistics, and correlation coefficients. 


Sample 


The total sample of this research consisted of two classes made up of 58 students 
from the Gymnasium in Prijepolje. According to the Curriculum (The Institute for the 
Advancement of Education, 2013), the students of this school attend chemistry classes 
during their four-year schooling. According to the Curriculum, general chemistry is 
studied in the first grade, inorganic chemistry with laboratory exercises in analytical 
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chemistry is studied in the second grade, organic chemistry is studied in the third grade, 
while the basics of biochemistry are studied in the fourth grade. 
The students who participated in this study attended the first grade and were aged 
15-16. According to the grades in chemistry at the end of the first semester, the structure 
of the students who were included in the sample for this research was the following: 
e 18.96% of the respondents had a grade 2 in chemistry at the end of the first 
grade, 
e 32.76% of the respondents had a grade 3 in chemistry at the end of the first 
grade, 
e 37.93% of the respondents had a grade 4 in chemistry at the end of the first 
grade and 
e 10.35% of the respondents had a grade 5 in chemistry at the end of the first 
grade. 
The respondents belonged to the urban population of different socioeconomic 
statuses, and they voluntarily joined the research. All students voluntarily participated in 
the research. Informed consent was obtained from students and school administration. 


Instrument and Procedures 


The Knowledge Test was specially designed for this research. The students had 
45 minutes (one school class) to solve the test. The respondents studied all the concepts 
present in the test tasks in chemistry classes. The test created for this research contained 
7 tasks. The students got one point for each correct task, so the maximum possible score 
on the test was 7 points. The incomplete tasks were not taken into consideration. 

In addition to the achievement, the test also measured the mental effort that a 
student invests while solving the task. The assessment of the invested mental effort was 
measured by a subjective technique with the application of the 7-point Likert scale. After 
each completed or uncompleted task, the students were asked to assess their mental 
effort by selecting an appropriate descriptive grade on the scale. During the statistical 
analysis of the results, the descriptions were numerically coded from "extremely easy" — 
numerical value | to "extremely difficult" — numerical value 7. 

To ensure objectivity in the estimation of cognitive complexity, the Table for 
the assessment of the difficulty of concepts and the estimation of their interactivity in 
problem tasks with a limiting reactant has been developed. The difficulty of the concepts 
is also estimated by the Rubric developed by Knaus et al. (2011). Concepts are rated 
according to their difficulty as easy, medium, or difficult. Table 1 describes the Table 
for estimating the difficulty of the concepts and their interactivity in the stoichiometric 
problems with a limiting reactant. 
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Table 1 
The Table for the Assessment of the Difficulty of the Concepts and the Estimation of 
Their Interactivity in the Stoichiometric Tasks with a Limiting Reactant 





A CHEMICAL EQUATION WITH GIVEN QUANTITATIVE RELATIONS 





The ratio of the reactants, the ratio between the given and the required substance is 1: 1 Easy 





The ratio of the reactants, the ratio between the given and the required substance is 1:X (X22) | Medium 





The ratio of the reactants, the ratio between the given and the required substance is X: Y (X22; 























Y=2 X#Y) Difficult 
A LIMITING REACTANT 

The reactants, the given and the required substances are given in a formula about the amount E 
of substance as 
The reactants, the given and the required substances are given in the units of mass Medium 
The reactants, the given and the required substance are given in the ratio amount: mass or Difficult 
mass: amount 

THE INTERACTIVITY OF THE CONCEPTS 
The task contains up to 2 concepts 0 
The task contains 3 concepts 1 
The task contains over 3 concepts 2 





The concept of a chemical equation with given quantitative relationships was 
chosen because the understanding of quantitative relationships among the participants in 
a chemical reaction is a key step in solving stoichiometric problems (Robinson, 2001). 
This problem is particularly pronounced if the quantitative relationship between the given 
and the required substance in the chemical equation is not 1: | (Dahsah & Coll, 2007; 
Hanson, 2016). Students sometimes choose the ratio 1: 1 in the problems with a limiting 
reactant regardless of the ratio of the stoichiometric coefficients of the participants 
in a chemical reaction, (Olmsted, 1999). If the given and the required substance are 
expressed in the same physical entity, the concept is considered easy (Robinson, 2001). 
If the given and the required substance are given in the same physical entity in the Rubric 
for the rating of the concepts that are present in stoichiometric problems, the concept 
is considered easy, regardless of whether the substances are expressed as the amount 
of substance, a mass, the number of particles, etc. (Horvat et al., 2016). However, it is 
documented that students achieve better results if the given and the required substance 
are expressed in the units of the amount of substance than if they are expressed in the 
units of mass (Astudillo & Niaz, 1996). So, the concept of a limiting reactant is the 
easiest if the substances are expressed in moles. A concept is a medium difficulty if the 
substances with which the calculations are made are given in the units of mass, and 
difficult if the given or the requested substance is expressed in the unit of quantity and 


the other substance is expressed in the mass unit. 
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Interactivity is evaluated based on the number of concepts in the task. If one 
concept is represented in the task, interactivity is evaluated with a value of 0. If the task 
contains two concepts, interactivity has value 1, and if the task contains three or more 
concepts, interactivity is evaluated by a numerical value 2. 


Data Analysis 


The quality of the test was evaluated by pre-test and post-test quality assurance 
parameters. Pre-test quality assurance parameters are determined by experts, whose 
narrow field of study is the methodology of chemistry teaching, as well as chemistry 
teachers in secondary schools. The test was assessed as valid based on the compliance of 
the tasks with the valid curriculum and the recommended textbooks. The experts assessed 
the tasks on the Test as diverse, with clearly defined requirements and meaningful 
sentences in compliance with language rules. Post-test quality assurance parameters 
are defined as basic statistical parameters: reliability coefficients, task discrimination 
indexes, the index of the discriminative property of a test, task difficulty indexes and test 
difficulty indexes. 

The obtained results were analyzed using IBM SPSS Statistics 22 software 
program. 


Research Results 


Tasks of different levels of cognitive complexity were used in this research. The 
complexity of the tasks depends on the concepts which are present in the task and defined 
in Table 1. All tasks are complex problem-solving tasks that include two sub-problems: 
determining the limiting reactant and calculating the required substance. The principle 
of using this Table is simple and objective. The procedure for calculating the rating of 
cognitive complexity is presented in the following example (Task number | in the Test): 
7.3 grams of hydrogen chloride and 4 grams of ammonia were mixed in the reaction 
vessel. The mentioned gases react according to the following reaction: 

HCl, + NH, NH,C1.,, 
Calculate how many grams of ammonium chloride is obtained in this reaction. 

The first phase involves determining the limiting reactant and the second phase 
involves calculating the mass of the formed product. The task contains the four concepts 
presented in Table 1. In the first phase, the limiting reactant is determined based on 
the data from the text of the task and the chemical reaction equation. The quantitative 
ratio between the reactants is 1: 1, and the reactants are given in the units of mass. 
The limiting reactant is determined based on the chemical reaction equation and the 
molar masses of the reactants. Using the Table for assessing the difficulty of the concepts 
and their interactivity in stoichiometric tasks with limiting reactants, their difficulty is 
estimated. The following concepts are present in this task: 

e The stoichiometric ratio of the reactants is 1:1 - The chemical equation concept 

with the given quantitative relationships which is "easy"; 

e The stoichiometric ratio of the reactant to the reaction product is 1:1 - The 

concept of a chemical equation with the given quantitative relationships which 
is "easy"; 
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e The reactants are expressed in the units of mass - The concept of a limiting 
reactant that is "medium" difficulty; 

e The reactant (the given substance) and the reaction product (the required 
substance) are expressed in the units of mass — The concept which is "medium" 
difficulty; 

e The task contains 4 concepts — Interactivity has a value of 2. 

Using the Rubric developed by Knaus et al. (Knaus et al., 2011), gives the total 

numerical rating of cognitive complexity of task number | in the test, which is 7. 


Table 2 
Numerical Rating of Task Number 1 

















The difficulty of the concept Number of concepts Numerical rating 
Easy 2 2 
Medium difficult 2 3 
Interactivity: 2 
Total rating of cognitive complexity: 7 





As in this case, the numerical rating of the cognitive complexity of other test tasks 
is calculated. The values are shown in Table 3. 


Table 3 
Numerical Rating of Cognitive Complexity of Test Tasks 





Task number Anumerical rating of cognitive complexity 
7 
7 
8 
6 
10 
10 
9 























NN} OD] Or] &] OO] NM] — 





The test used in this study showed good metric characteristics. Reliability was 
calculated as a measure of internal consistency and expressed as a Cronbach’s coefficient 
a which valued .65 for the achievements, and .73 for self-estimated mental effort, which 
indicates good reliability (Jonsson & Svingby, 2007). The task's difficulty index ranges 
from 13.79% to 96.55% (the average value is 49.51%, which makes the test moderately 
difficult). The value of the difficulty index of two tasks is less than 25%, which makes 
them difficult, whereas the difficulty index value of one task is higher than 75%, which 
makes it an easy task (Towns, 2014). The values of the discrimination index range from 
.07 to 1 (the medium value is .59 which represents an excellent discrimination index 
of the test). Six tasks have an excellent index of discrimination which is higher than .4 
(even 3 tasks have an index of discrimination higher than .94), whereas only one task has 
a poor index of discrimination (.07), so it should be revised for future use. 
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The basic statistical parameters of the test are shown in Table 4. 


Table 4 
Descriptive Statistics for the Students' Performance and Mental Effort 





Students’ ratings 























, 4 = 

Parameter Students’ performance’ (N=58) of mental effort? (N=58) 
Average 3.46 3.80 

Standard deviation 1.71 .90 

Standard skewness -.12 67 

Standard kurtosis -1.03 12 

Minimum 0 1.86 

Maximum 7 6.14 

Range 7 4.28 





‘Students' performances could range from 0 to 7.Possible ratings for invested mental effort could range from 1 to 7: extremely 
easy (1) to extremely difficult (7) 





The values of the standard skewness and kurtosis for the performance and the 
invested mental effort indicated that in both cases a normal distribution was presented. 
Additional Shapiro-Wilk’s test did not confirm the assumption of the normal distribution 
of students’ performances (F=.20; p<.05) and students’ mental effort (F=.16; p<.05). 

The validation of the test was performed by Spearman’s correlation which refers 
to the observation of the relation between the students’ performances and self-estimated 
mental effort. The graphic dependence and the statistical parameters of regression 
analysis are shown in Figure | and Table 5. 


Figure 1 
The Correlation of the Students’ Performance with the Students' Ratings of the 
Invested Mental Effort 
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Table 2 
The Statistical Parameters of the Regression Analysis of the Students’ Performances 
and the Students' Ratings of the Invested Mental Effort 











Parameter Value 
The correlation coefficient -.58 
p-value .02 
Equation Performance = 1.67 - .31xME 





This dependence describes a moderate correlation (r=- .58; p=.02). P-value is 
less than .05, which indicates that there is a statistically significant correlation between 
measures of mental effort and performances at the 95% confidence level. The existence 
of a significant correlation between performance and mental effort has already been 
confirmed in the studies about the dimensioning cognitive complexity of the problems 
(Knaus et al., 2011; Raker et al., 2013; Horvat et al., 2016; 2017; 2020). 

Procedures for assessing the cognitive complexity of problem tasks are usually 
validated by combining with measures of students' performances and measures of mental 
effort (Knaus et al., 2011; Raker et al., 2013; Horvat et al., 2016; 2017; 2020). 

In the first phase, the regression analysis of the dependence of students’ 
performances on the estimated cognitive complexity. As the distribution of performances 
and mental effort did not satisfy the normal distribution criterion and since the performance 
values can only be zero or one, it was done with biserial correlation analysis. The results 
of the analysis are tabulated in Table 3. The dependence of the numerical rating of the 
cognitive complexity from the students’ performances on test tasks (406 items) was 
observed. 


Table 6 
Statistical Parameters of the Regression Analysis of Students’ Performance and the 
Cognitive Complexity 














Parameter Value 
The correlation coefficient - 37 
p-value p<.001 
Equation Achievement = 1.64 - .13xCC 














The coefficients obtained by the regression analysis (7, = -.37; p < .05) indicate 
that there is a moderate but statistically significant correlation between the dependent 
and the independent variables (Evans, 1996). The negative value of the correlation 
coefficient shows that the increase in the cognitive complexity of the problem results in 
performance decreases. 

The second phase of validation of the procedure for the assessment of cognitive 
complexity of the tasks with limiting reactants is a correlation analysis of the dependence 
of self-invested mental effort on the rating of the cognitive complexity of the tasks. 
Since mental effort does not satisfy normal distribution, the Spearman’s p correlation 
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coefficient was determined. Statistical parameters and graphic dependence are shown in 
Table 7. 


Table 7 
Statistical Parameters of the Regression Analysis of the Student's Self-invested 
Mental Effort and the Cognitive Complexity 











Parameter Value 
The correlation coefficient 04 
p-value p<.001 
Equation Mental Effort = -.35+.47xCC 





The correlation coefficient (7, =.54) and the p-value (p<.05) indicate a moderately 
strong statistically significant correlation between the mental effort as a dependent 
variable and the numerical value of the cognitive complexity rating as an independent 
variable. 


Discussion 


The test used in this research had good statistical parameters. The average 
achievement of students was 3.46, which means that students could solve a minimum of 
three tasks. The average value of students’ mental effort on the test was 3.80. According 
to the 7-point Likert scale, the test was “neither easy nor difficult”. 

The validity of the procedure was confirmed with a correlation between students’ 
performances and the average value of students’ self-invested mental effort from the 
numerical rate of cognitive complexity. The positive value of the coefficient of correlation 
indicates that students invest a higher mental effort to solve the task with increasing 
cognitive complexity. At the same time performance decreases. This is consistent with 
the results of Pollock et al., (2002). Since the information is selectively processed during 
the process of solving the task, it is necessary to optimize its cognitive complexity when 
designing the task (Halford et al., 1998). Cognitively more complex tasks impose a 
greater mental load on the working memory of respondents and condition lower student 
achievement (Campbell & Gingrich, 1986). 

The rating of the cognitive complexity of the examined tasks ranges between 6 
and 10. The first and second tasks have the value 7 of cognitive complexity, where the 
average achievement is .97 and .53. A strikingly significant difference in the achievement 
is the consequence of the strategy for solving stoichiometric tasks that is favorable in 
the used textbook literature which favors the comparison of masses (Nikolajevié & 
Surjanovié, 2015). Namely, in the first task, the reactants are expressed in mass units, 
and in the second task, their amounts in moles are given. A similar situation has been 
noticed in two of the most difficult tasks. The rating of the cognitive complexity of the 
fifth and sixth tasks is the same - 10. The average students’ achievements in these tasks 
are .69 and .22. And in this case, student achievement is the highest in the fifth task in 
which the participants of the reaction are expressed in the mass units. It can be noted 
that students manipulate the mass more effectively than the moles in tasks with limiting 
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reagents, which is unexpected. Olmsted (1999) noted that students solved problems with 
a limiting reactant with the strategy used by the teacher, or that is recommended in 
textbooks. Namely, the students insisted on comparing the mass of the reactants rather 
than on comparing the amounts of the substances. 


Conclusions and Implications 


The basic research task was to create a procedure for assessment of the cognitive 
complexity of stoichiometric tasks with a limiting reactant. To accomplish this, a Table 
for assessing the difficulty of the concepts and their interactivity was constructed. Thus, 
a valid procedure was obtained for assessing the cognitive complexity of stoichiometric 
problems with a limiting reactant. 

The validity of the Procedure was confirmed by a series of regression analyses 
of the dependence of students’ performances and their assessment of invested mental 
effort from the cognitive complexity. High values of correlation coefficients were 
determined: -.37 for the dependence of achievement-cognitive complexity and 0.54 for 
the dependence of mental effort-cognitive complexity. The results of these analyses are 
presented in graphical and tabular form and the form of equations of regression analysis. 

The process developed in this research should help teachers in the design of the 
tasks with a limiting reactant of different levels of cognitive complexity. By gradually 
developing the complexity of problem-solving tasks with a limiting reactant, the teacher 
can develop problem-solving strategies, taking care of the mastered concepts. This can 
affect the cognitive development of each student. The design of the tasks of different 
levels of cognitive complexity is a way to estimate the learning outcomes better and to 
determine cognitive load using the measures of mental effort. 

The limitation of this survey is a sample of respondents. Also, when literature is 
concerned, a small number of stoichiometric tasks with limiting reactants are present in it 
and they are exclusively expressed in the units of mass. Therefore, students' mistakes and 
increased amount of their mental effort come from an algorithmic approach to solving 
these problems, which also arise from the teacher's strategy. 

The implications for further research in this area are the design of problem-solving 
assignments in which the reactants will be expressed, in addition to the units for mass 
and the quantity of substance, in another unit (the number of particles, volumes, etc.). 
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Abstract 


The Lawson classroom test of scientific reasoning is a quite popular and widely used tool that 
measures the level and development of the student's scientific reasoning skills. In this contribution, 
the results of this test for the N=446 students of the Faculty of Science Palacky University Olomouc 
from the years 2018-2020 at the beginning of their study were analysed. Calculation of the 
standard characteristics of the test items (difficulty index, discrimination index and point biserial 
coefficient) confirms that the test meets the required criteria for the included questions and its 
overall consistency. Performing the Mann-Whitney rank test with a standard significance level a = 
.05, the statistically significant difference between males and females was found, on the other 
hand, no statistically significant difference between the students preparing for their prospective 
teacher careers and those with technical specializations in chemistry or physics was identified. 
This contradicts a common belief that students choosing teacher preparation programmes are 
generally worse in scientific performance and technical subjects. The results also show that about 
a quoter of the students come to university with a low level of their scientific reasoning skills 
which should be therefore supported and further developed in lecturing and teaching. 

Keywords: scientific reasoning, test evaluation, Lawson Classroom Test of Scientific Reasoning 


Introduction 


In science, technology, engineering, and mathematics education (STEM) there 
has been increased emphasis not only on the learning of content knowledge but also 
on the development of skills connected with the skills important for practical science, 
including scientific reasoning. Generally, scientific literacy is currently considered one 
of the central goals for the development of 21st-century citizens and scientific reasoning 
ability is determined as an important factor for fostering student performance in science 
learning and has gained increasing attention from science educators and researchers. 
Moreover, scientific reasoning represents a hidden variable that can influence students’ 
understanding of learning new concepts. Therefore, mapping the level of scientific 
reasoning skills within a class or a course may be useful to reveal students who have 
more difficulty in learning and may provide the information to develop an appropriate 
method which overcomes the pedagogical issues. 

In the literature, there are many definitions of scientific reasoning. Following 
(Bao et al., 2009a) we accept the position that scientific reasoning represents the 
cognitive skills necessary to understand and evaluate scientific information, which often 
involves understanding and evaluating theoretical, statistical, and causal hypotheses. 
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Broadly defined, it includes the thinking and reasoning skills involved in the inquiry, 
experimentation, evidence evaluation, inference, and argumentation that support the 
formation and modification of concepts and theories about the natural and social world. 

In 1978, Lawson designed an assessment instrument that measures students’ 
level of scientific reasoning development (Lawson 1978). The paper and pencil style 
addressed the need for a reliable, convenient tool enabling to examine larger numbers of 
respondents that would be more practical for classroom use, compared to the Piagetian 
tasks. In 2000, building on previous work, Lawson developed an improved version of 
the assessment instrument, named Lawson’s Classroom Test of Scientific Reasoning 
(LCTSR). It is a two-tier, multiple-choice test with 24 items (Lawson, 2000). A two- 
tier multiple-choice item pairs (questions 1—22) contain a question with some possible 
answer choices, followed by another question proving some possible reasons for the 
response to the previous question. According to (Zhou et al., 2021), a correct answer 
with incorrect reasoning indicates an intermediate level of reasoning development, 
whereas an incorrect answer with correct reasoning is likely a result of guessing. All 
the answer choices were designed based on previous studies on student misconceptions 
with free response tests, interviews, and relevant literature (for a more detailed recent 
review of the LCTSR development see e.g. (Zhou et al., 2021)). After more than 20 years 
we may conclude that the LCTSR is a practical tool assessing a unidimensional scale 
of scientific reasoning with good overall reliability, though inspections of individual 
question pairs revealed some validity concerns for several question pairs (Bao et al., 
2018). Nevertheless, correlations were calculated between LCTSR scores and other 
measures of reasoning, and adequate values found. The Lawson test gained popularity 
among STEM educators and researchers, so far it has been given to several thousands 
of students and results were published in tens of papers, some of which are also referred 
to in this text. It is appropriate for a wide spectrum of contexts, namely intro colleges 
(which is also our case), high schools and middle schools. 

In this research, the LCTSR, which assesses students’ abilities in six dimensions 
including conservation of matter and volume, proportional reasoning, control of variables, 
probability reasoning, correlation reasoning, and hypothetical-deductive reasoning, was 
also used. All of the above-mentioned skills were identified as necessary for a successful 
STEM career. 

The LCTSR scores at the beginning of the course may tell us a lot about our 
students’ initial reasoning levels so that we get a better sense of which types of reasoning 
our students need more help with and can adjust appropriate teaching methods. According 
to the scores divided into 3 equal intervals, Lawson suggested classifying students into 
three formal reasoning categories: concrete operational (scoring up to 8 points, ie., 
33%), transitional (scoring between 9 and 16 points, i.e., 34% and 67%), and formal 
operational (scoring above 16 points, i.e., 67%). For the comparison of the score results, 
it is important to mention some other similar studies. For instance, (Bao et al., 2009a) 
report an average score of 75% based on the VN = 5760 freshmen science and engineering 
students in four U.S. and three Chinese universities of medium ranking. On the contrary, 
(Sriyansyah & Saepuzaman, 2017) found among 29 first-year prospective physics 
teachers who attended introductory physics courses in 2015 at Indonesia University of 
Education no formal reasoner and 31% concrete reasoners. 

A strong positive statistically significant correlation between students’ pre- 

| instruction scores for reasoning ability (measured by the LCTSR) and some tools 
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measuring the conceptual knowledge in a particular field of physics has been reported in 
high school and university introductory courses, e.g., learning of forces and Newtonian 
laws (measured by the Force Concept Inventory) and basic concepts of electricity and 
magnetism (measured by Brief Electricity and Magnetism Assessment); see (Nieminen 
et al., 2012; Pyper, 2011; Sriyansyah & Saepuzaman, 2017). For the original Lawson 
1978 test administered to secondary school students in Israel and the U.S.A. there was 
only a small correlation between achievement in science and maths and the Lawson 
test; and the Israeli population achieved significantly higher than the US students on the 
Piagetian skills measured by the test (Hofstein & Mandler, 1985). Also, (Maloney, 1981) 
reported statistically significant differences between the science-major and non-science 
major students (in the original Lawson 1978 test), with all six students with the lowest 
scores (concrete operational level) dropping the intro-physics course without completing 
exams. Similarly, (Moore, & Rubbo, 2012) have found that non-STEM (science, 
technology, engineering, and mathematics) majors taking either a conceptual physics 
or astronomy course at two regional comprehensive institutions scored significantly 
lower on the LCTSR in comparison to national average STEM majors (the majority of 
their non-STEM students can be classified as either concrete operational or transitional 
reasoners, whereas in their STEM population formal operational reasoners are far more 
prevalent). Scores on the LCTSR were then correlated with normalized learning gains 
on various concept inventories. The correlation was strongest for content that can be 
categorized as mostly theoretical, meaning a lack of directly observable exemplars, 
and weakest for content categorized as mostly descriptive, where directly observable 
exemplars are abundant. These results further demonstrate that differences in student 
populations are important when comparing normalized gains on concept inventories, and 
the achievement of significant gains in scientific reasoning requires a re-evaluation of the 
traditional approach to physics for non-STEM students. 

Another question is, to which extent the scientific reasoning skills are related 
to students’ performance and are supported within university introductory courses. 
For instance, (Bao et al., 2009a, Bao et al., 2009b) found that the rigorous learning 
of physics knowledge in middle and high schools makes a significant impact on the 
ability of students in China to solve physics problems, but this knowledge does not 
seem to have a direct effect on their general scientific reasoning ability. This suggests 
that current education and assessment in the STEM disciplines often emphasize factual 
recall over a deep understanding of scientific reasoning. Similarly, based on the results 
from two Chinese universities (Ding, 2013) shows that, regardless of their major, student 
reasoning skills measured by the LCTSR remained largely constant across the four years 
of higher education in accordance with (Pyper, 2011) and his data from several years 
and several different classes that have shown that Lawson test scores do not change 
much over the course of a single semester. Students of average and lower reasoning 
abilities struggle in solving problems that depend on conceptual understanding and may 
depend more readily on memorization of simple procedures to solve problems (Fabby 
& Koenig 2015). As pointed by (Piraksa, Srisawasdi & Koul, 2014), there is a critical 
area for improvement of students’ scientific reasoning ability. In biology (Susilowati & 
Anam, 2017) successfully verified the 5E-learning model (Engagement, Exploration, 
Explanation, Elaboration, Evaluation) to be effective in scientific reasoning and problem- 


solving ability of students. 
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This all indicates, that LCTSR can provide valuable information and feedback for 
the course instruction, at least in some context. Therefore, it makes sense to ask, whether 
the results can be affected by some parameters, like gender or study programmes of the 
respondents. 


Research Methodology 


The LCTSR was tested with the students of the Faculty of Science, Palacky 
University Olomouc within the years 2018-2020 with the total number of N = 446 
respondents (158 in 2018, 133 in 2019 and 155 in 2021) including chemistry-major 
and physics-major students (technical programmes, called non-teachers further in the 
text) and students in prospective science teachers’ programmes (called teachers below). 
Let us remark that in the Czech Republic future teachers are typically specialized in 
two-subject combinations like e.g., maths-physics or chemistry-biology. The test was 
typically taken within introductory courses in the first week of the semester, thus the 
scores were not affected by any university-level instruction and rather correspond to the 
level of scientific reasoning at the end of their high schools. To motivate the students to 
think about questions seriously and not just randomly mark the answers, we gave them 
some extra points for the overall scores of the courses within which we have organized 
the testing. The organizing and scoring the test also followed the recommendations 
and instruction provided for LCTSR by PhysPort network support developed and 
maintained by the American Association of Physics Teachers (https://www.physport. 
org/assessments/). 

After the scoring, the standard characteristics for test items and the whole test were 
determined as described below within the results sections. The standard characteristics 
of the test items and the test as a whole were determined according to (Ding et al., 
2013; Eaton et al., 2019) where the corresponding formulas and recommended ranges 
are introduced. The item difficulty index P is a measure of the difficulty of a single test 
question. It is calculated by taking the ratio of the number M1 of correct responses on the 
question to the total number N of students who attempted the question (P = N1/N). The 
recommended values are in the interval <.3, .9>. 

The item discrimination index D is a measure of the discriminatory power of 
each item in a test, it measures the extent to which a single test item distinguishes students 
with the better total scores well from those with lower ones. The possible range for the 
item discrimination index D is <-1, +1> with negative values for the questions answered 
better by the students with lower total test scores. There are more ways how to determine 
the groups with higher and lower scores based either on median or quartiles. In our case, 
following (Ding et al., 2013), we divided the whole sample of students according to 
quartiles and proceed with two different groups of equal size, a high group H and a low 
group L (using the top 25% as the high group and the bottom 25% as the low group). For 
a specific test item, one counts the number of correct responses in both H and L groups: 
namely, N,, and N_. If the total number of students who take the test is N, then D = (N,,— 
N_)/(N/4). An item is typically considered to provide good discrimination if D > .3. 

The point biserial coefficient (sometimes referred to as the reliability index for 
each item) is a measure of the consistency of a single test item with the whole test. It 
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reflects the correlation between students’ scores on an individual item and their scores 
on the entire test and is basically a form of the correlation coefficient. The point biserial 
coefficient has a possible range of <-1, +1>. Ideally, all items in a test should be highly 
correlated with the total score, but that is somewhat unrealistic for a test with a large 
number of items. The criterion widely adopted for measuring the “consistency” or 
“reliability” of a test item is rpbs > .2 (Ding et al., 2013). 

The Kuder-Richardson reliability index is a measure of the self-consistency of a 
whole test without administering one test twice or designing two tests, 1.e., by using the 
information from one test administered just once. Using the formula KR-21 (Ding et al., 
2013), possible values for the KR-21 reliability index fall into the range <0,1>. Different 
tests for various purposes have different criteria. A widely accepted criterion is that tests 
with a reliability index higher than .7 are reliable for group measurement and tests with 
a reliability index higher than 0.8 are reliable for individual measurement. Under most 
circumstances in physics education, evaluation instruments are designed to be used to 
measure a large group of students, so if a certain physics test has a reliability index 
greater than .7, one can safely claim it is a reliable test. 


Figure 1 

Histogram of Total Test Scores with a Density Estimate to Smooth the Distribution 
Shows that Nearly One Quarter of the Students Fall into the Lowest Piagetan Level 
of Concrete Operational Reasoning 
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Ferguson’s delta is another whole-test statistic measuring the discriminatory power 
of an entire test by investigating how broadly the total scores of a sample distribute over 
the possible range. Ifa test is designed and employed to discriminate among students, one 
would like to see a broad distribution of total scores. The possible range of Ferguson’s 
delta values is <0,1>. If a test has Ferguson’s delta greater than .9, the test is considered 
to offer good discrimination. 

The scores in the six dimensions of scientific reasoning covered by the LCTSR 
were compared and a correlation matrix between them found to identify, which of 
them are most important for the overall test ranking. Based on the obtained results two 
hypotheses were tested — first, that the males over-score females and second, whether the 
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students choosing the science teacher preparation study programmes reach a different 
level of scientific reasoning. Both hypotheses were checked by the Mann-Whitney U 
test with a significance level a = .05 and a two-sided f¢-test for identical means with 
the less strict assumption of different variances. The data and results from the years 
2018 and 2019 were studied in a bachelor thesis, later the data from the autumn 2020 
were included and previous conclusions confirmed. The statistical calculations were 
performed by the Python programming language using its open-source data analysis and 
manipulation library pandas and its module for statistical functions scipy.stats. 


Research Results 


The histogram of LCTSR scores for all NV = 446 students is in Figure 1. The 
corresponding average is 12.7 (52.8% of the possible maximum), median 13 with a 
standard deviation of the distribution 5.13. It can be seen that nearly one quarter of 
the students entering our courses (24.7%) rank into the lowest category of concrete 
operational reasoning and approximately the same part (24.9%) into the highest category 
of formal operational, the rest (50.4%) belong to the transitional category. Only one 
student reached the full score of 24 points. In the context of the Czech Republic, among 
prospective physics teachers in the first year of their study at the Faculty of Mathematics 
and Physics, Charles University, Prague in 2011 there were only 9.3% concrete reasoners, 
and 59.3% students reached the highest level of formal operational reasoning. Therefore, 
our proportion of the students in the low concrete operational category is surprisingly 
high which may indicate that some students in this category could face problems in 
learning scientific concepts. 


Figure 2 
Item Characteristics for All 24 Questions in the LCTSR: Difficulty Index P. Discrimination 
Index D and Point Biserial Coefficient r,,. 
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The density estimate to smooth our LCTSR scores distribution in Figure 1 shows 
the asymmetry in the direction of lower scores, which also confirms the negative skewness 
of the distribution — 0.0386. A negative kurtosis —0.817 means that the distribution is 
flatter than a normal distribution with the same mean and standard deviation. 

The question-item measures are summarized in Figure 2. With the average 
difficulty index P= .623 and the minimum value .343 for question 12, we can conclude 
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that there are no extremely difficult questions with P < .3 in LCTSR. The maximum 
P values 0.930 and .924 belong to questions 1 and 2 respectively; those two easiest 
questions dealing with the conservation of mass are out of the suggested range, and 
we can mark them as too easy. Question 12 belongs to the pair of questions combining 
identification and control of variables and probabilistic thinking as a second question 
providing a reason for the answer to question 11. 


In our case, the average discrimination index D = .630, with a minimum value 
.305 for questions 1 and 2 with the highest P values; the easiest questions naturally have 
the lowest discriminatory power (but still D > .3). The highest values of D correspond 
to questions 5 (D = .969) and 18 (D = .996) which represent proportional thinking 
and advanced probabilistic thinking, respectively. Similarly, our average point biserial 


coefficient T pbs = 400 is satisfactory, even the lowest values .212 and .205 for the easiest 
questions | and 2 respectively are above the limit .2. 

The Kuder-Richardson reliability index value .847 obtained for LCTSR is again 
compatible with the criterium to be over .7 and shows the test can be considered as 
reliable. Obtained value for the Ferguson’s delta .985 is also satisfactory (> .9). 

In Figure 3 we can see the scores evaluated according to the categories of 
scientific reasoning and the correlation matrix among them and the total test scores. The 
most successful is the conservation of mass (92.2%), which corresponds to the easiest 
questions | and 2 as discussed above. Not surprisingly, the lowest scores correspond to 
hypothetico-deductive thinking and correlation thinking (39.9% and 42.4% respectively). 
It is in partial agreement with the Thailand study (Piraksa et al., 2014), in which the 
lowest scores correspond to hypothetical-deductive reasoning, control of variables and 
proportional thinking. 


Figure 3 

Scores in the components of scientific reasoning defined by Lawson (in percent 
to the maximum possible scores in those parts) and the corresponding correlation 
matrix (CM — Conservation of mass, CV — Conservation of volume, PR — Proportional 
reasoning, ICV — Identification and control of variables, PT — Probabilistic thinking, 
CT — Correlation thinking, HDR — Hypothetico-deductive reasoning, Total — Total test 
score) 
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Also, (Moore & Rubbo, 2012) reported their non-STEM students to demonstrate 
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significant difficulty with proportional and hypothetico-deductive reasoning. From 
our correlation matrix in Figure 3, we can see, that the mutual correlation among the 
scores in the categories are loose (all Spearman correlation coefficients below .5), a 
little bit stronger correlation can be found between the total test scores and identification 
and control of variables category, probabilistic and proportional thinking (Spearman 
correlation coefficients above .6). Recently, it has been shown by (Hejnova et al., 2018), 
that for the 14-15-year students the ability to use the existing knowledge strongly 
correlates with three dimensions of the scientific reasoning structure: proportional 
reasoning, control of variables and probability thinking, which demonstrates the highest 
correlation with the total test scores in our research. 

Working with the test scores we noticed that there could be a significant difference 
in the performance of male and female students (see the box plot in Figure 4), which was 
also indicated in the data we privately obtained from the Faculty of Mathematics and 
Physics, Charles University, Prague. Among our respondents, there were 286 females 
with a mean score 11.5 (median 11, minimum 1, maximum 22) and 160 males with 
a mean score 14.9 (median 15, minimum |, maximum 24). Mann-Whitney U test for 
the distributions and f-test of the scores for the mean score differences confirmed, that 
the difference is statistically significant (p-value .000). The results of other studies are 
not unequivocal in such a conclusion. For the original Lawson 1978 test administered 
to secondary school students in Israel and the U.S.A. was also found that, in general, 
males outperformed females (Hofstein & Mandler, 1985). On the other hand, the 
results of the Thailand LCTSR study (Piraksa et al., 2014) did not indicate that gender 
should significantly impact students’ scientific reasoning ability. Similarly, the results of 
Indonesian research (Novia et al., 2018) showed that no statistical difference between 
male and female scientific reasoning mean scores were observed and that gender and 
age do not significantly impact students’ scientific reasoning ability in the first year 
of university education. Thus, besides the fact the LCTSR was designed by a male, 
the gender difference in the mean scores may also have roots in the local cultural and 
historical backgrounds. 


Figure 4 

Box Plots Showing the Quartiles of the Datasets and the Whisker of the Dataset 
Extends for the Total Scores with Respect to Gender and the Kind of a Study 
Programme of the Respondents 
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Another important question we wanted to answer relates to a common belief and 
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opinion, that the carrier of prospective teachers is mostly chosen by students with worse 
high school results who give up or fail to proceed with more demanded study programs 
(e.g., law, economy, medicine, engineering, or technical science programs). Therefore, 
we checked whether there is a significant difference between “future teachers” and 
“non-teachers” in scientific reasoning skills measured by LCTSR. In our sample, the 
156 prospective teachers reached a mean score 13.1 (median 14, minimum 3, maximum 
23), 290 non-teachers mean score 12.5 (median 13, minimum 1, maximum 24). Both 
the Mann-Whitney U test (p-value .154) and the ¢-test for the different mean scores 
(p-value .234) confirmed, there is no statistically important difference in the distributions 
of the scores for those two groups. Thus, we can conclude, that at least within our Faculty 
of Science, the students of science education programmes should not be automatically 
taken as worse or with lower prerequisites to acquire, learn and understand scientific 
concepts and in solving problems. We certainly realize that might be rather a faculty- 
specific conclusion, and we cannot compare it with any similar study at this point. 


Conclusions 


It was demonstrated and verified the LCTSR is a relatively reliable tool fulfilling 
the demanded values for item characteristics like item difficulty index, discrimination 
index and point biserial coefficient (maybe except for the easiest questions 1 and 2, which 
may be considered as motivating). Also, both the Kuder-Richardson reliability index and 
Ferguson’s delta measuring the test self-consistency and overall discrimination power 
meet usual criteria. We were also able to identify the identification and control of variables 
category, probabilistic thinking, and proportional reasoning as the key dimensions of the 
scientific reasoning correlating most with the total test scores. In our sample, there is 
a Statistically significant gender bias of the score distributions, but students entering 
science education programmes achieved a comparable mean level of scientific reasoning 
as their colleagues enrolling in technical-scientific programmes. 

The open problems for some further research should answer the relation between 
the LCTSR scores and possible learning gains within introductory courses, which is not 
generally straightforward. Also, it is not clear, to which extent scientific reasoning is 
developed within some types of courses. 

On the other hand, training in scientific reasoning may also have a long-term 
impact on student academic achievement. With respect to relatively low mean scores 
for our sample, there is evidently demanded to develop and strengthen the scientific 
reasoning also within the introductory university courses, as most of our students belong 
to the transitional reasoners' category. Our results agree with (Bao, et al. 2009a), that 
students start to fully develop the basic reasoning abilities around their college years. 
The take-home message might be that the sorts of activities that require students to 
develop, explain and defend their reasoning help them to develop cognitively in ways 
that lectures or even peer interactions in a lecture-like environment cannot provide 
much effectively. We cannot assume that our incoming students are able to reason in the 
sophisticated ways which they need to really understand abstract and complex concepts 
like force and energy. If we do really care that our students get these concepts, we need 
to provide explicit opportunities to improve their reasoning. Hence, it is necessary to 
make reasoning explicit in the instruction. Although the reasoning abilities tested 
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in the LCTSR may appear simple to expert scientists, these are crucial fundamental 
components for more sophisticated skills. However, to assess the reasoning abilities of 
senior college students and graduate students, we need to develop questions that involve 
more advanced reasoning components. This also implied that instructional pedagogy in 
science classrooms and university or college lecture rooms should be more emphasized 
on the way of teaching how to reason casually based on hypothesis generation, how 
to design well fair science experiments, and how to determine the correlation between 
target variables, to enhance the development of students’ scientific reasoning ability. 
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Abstract 


The newspaper headlines in July 2020, reflected the context of COVID-19 and the challenges 
in the education sector in South Africa. Pre-service teachers completing a Biological Sciences 
for Educations Research and Service-Learning module conducted their Service-Learning in 
their home contexts, which under normal times, they would do so in the neighbouring university 
contexts. The research question: Why did the Biological Sciences pre-service teachers' experience 
COVID-19 as an enabler for their Service-Learning projects. An interpretive, qualitative case 
study was adopted to explore the pre-service teacher s experiences of their projects undertaken. 
The data gathering methods included document analysis (pre-service teacher ’ reflective diaries); 
observation of module reflective sessions and seminar presentations and visual methodology 
(pre-service teachers made videos). The data analysis using descriptive content analysis. The 
research rigour of credibility and dependability were worked with, and the research ethics were 
considered. The results indicated that during the learning about the project, the pre-service 
teachers had emotional experiences of fear, excitement and even confusion. During the planning 
for the project, they had concerns about Covid-19 restrictions and access to placement sites, 
what to do, who to work with and the nature of the projects planned. The action of the Service- 
Learning indicated the collaboration and teamwork, imagination and creativity, including the 
contextually relevant problem-solving actions that were undertaken. Pre-service teachers were in 
their own communities where they excelled and built relationships and valued their community 
members. Service-Learning should be completed in the pre-service teacher ’s home contexts for 
greater relevance, value and connectedness with their community. 

Keywords: biological sciences, case study, COVID-19 pandemic, pre-service-teacher education, 
service-learning 


Introduction 


COVID-19 - Excess deaths point to hidden toll in South Africa; The primary 
goal of the South African COVID-19 response is to slow the spread of infections; South 
Africa has the world’s fifth-worst Epidemic, with 471,123 cases; South Africa’s R350 
coronavirus grants will be made available to more people; South Africa crosses 400,000 
mark; Schools closed amid virus. These newspaper headlines depict the context of South 
Africa during the pandemic. Reading these headlines and the articles linked to them is 
different to actually experiencing it directly, and being challenged by feelings of fear, 
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anxiety, hopelessness, including hope with growth for what can we do to influence the 
lives of communities in sustainable ways, and still expecting to be living a “normal” life 
as a teacher educator and working with pre-service teachers. 

In the School of Education, Community Engagement embraces the University 
of KwaZulu-Natal Strategic Plan 2017 — 2021, Goal 3 (Strategic Plan, University of 
KwaZulu-Natal, 2017). This goalis concerned with Community Engagement which should 
be high impact societal and stakeholder community engagement that has meaningful 
interactions for mutual benefit. In the Southern African Higher Education sector, the 
strategic outcome of Community Engagement is in regard to social responsiveness 
and to contribute to the development of students, communities and Higher Education 
Institutions. This was in response to the call of the White Paper on the Transformation 
of Higher Education (Department of Education, 1997). The principals identified in this 
policy document are social responsibility, humaneness and social justice, and they are 
viewed as pivotal to engaging with communities. 

In the context of South Africa and Higher Education, including Basic Education 
(schooling levels) the intensity of the pandemic was indicated by the newspaper 
headlines. Many issues experienced were linked to the extensive lockdown that was 
implemented by the government in March 2020. Since there was restricted movement of 
citizens and the closure of the universities and schools, students and learners experienced 
many challenges - access to the internet as online learning was implemented by the 
institutions that could afford it, lack of education resources and general attendance and 
access to learning, safety measures and the prevention of contracting the virus. Many 
learners (schooling level) and students (post schooling levels) were dealing with added 
trauma, including economic dislocation, hunger, and mental health challenges, all of 
which clearly affect learning, regardless of how it takes place. 

Pre-service teachers completing a Biological Sciences for Educators Research and 
Service-Learning module were challenged with deciding what to do and how to complete 
their projects in the context of the pandemic. This was particularly important for the 
service with communities to be recognised and the pre-service teachers to achieve credits 
for the work that they had completed. Service-Learning may be defined as experiential 
learning that integrates practical experiences into the academic curriculum (Furco, 2001), 
with the life purpose aspect (Opazo, et al., 2018). As a teacher education institution 
integrating community engagement in a module gave credence to the Scholarship of 
Teaching and Learning (SoTL) (Shulman, 2000) focusing on the expectations, planning, 
and the actions for the completion of the Service-Learning. The Biological Sciences 
module was taught virtually to third-and fourth-year pre-service teachers in 2020 and 
focused on Service-Learning, both were engaged with. The 16-credit module comprises 
three 90-minute lectures per week for 5 weeks and one 90-minute lecture and Service- 
Learning placement site work for three to four hours per week, for seven weeks. During 
COVID-19, many issues regarding students working in communities were raised: a 
focus on service-Learning (more on the learning and less on the service) and minimal 
contact with communities (Morton & Rosenveld, 2021); the new digital face of Service- 
Learning where academic efforts were aligned with community priorities (Brooks, 
2020). In the case of the pre-service teachers, due to COVID-19, they conducted their 
Service-Learning projects in their home communities, not in the areas around campus, in 
a mask to mask nature. So, a completely different contextual setting was to be engaged 
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with the for project completion. So, a question raised is, Why did the pre-service teachers 
experience COVID-19 as an enabler/enhancer for how they planned and implemented 
their Service-Learning? 


Research Methodology 
General Background 


An interpretive, qualitative, case study was undertaken to provide meaning and 
an in-depth understanding of the pre-service teachers’ experiences of conducting their 
Service-Learning during COVID-19. They were expected to be of service in a mask to 
mask setting with the community and the learning was in discussions and reflections 
during the virtual sessions. The pre-service teachers provided service activities that did 
not entail teaching Life Sciences at a school, it could be working with any community 
organisation, which is referred to as a placement site. So, they had to find suitable 
placement sites in their home communities. These sites included farming areas, local 
primary schools, Early Childhood centres, Children’s homes, police stations, old age 
homes, any organisation in their community that would accept them to be of service with 
the organisation. The nature of the service was decided in negotiation with the placement 
manager, taking the needs of the organisation and the competences of the pre-service 
teacher into account. 


Sample and Procedures 


The sampling method adopted was purposive, in that all the one hundred and 
seventy pre-service teachers, working in groups of two or three, with a total of 71 
projects, were included. All these projects were sampled to provide a descriptive account 
of the nature of the service that was undertaken, in the engagement with communities. 
For an in-depth analysis only two projects that were based in rural, primary education 
sectors were purposively selected. 

Since the module was presented in phases, the data gathering was also conducted 
within these: phase one was concerned with the expectations, informative and 
exploratory sessions at the beginning; phase two with the planning of the interventions, 
and phase three with the time spent working with the placement site managers. In each 
of the phases the pre-service teachers reflected on their thinking and feelings linked 
to the incidents that they experienced, document analysis of the pre-service teacher’s 
reflective diaries. The module, reflective sessions were integrated in the three phases as 
well and it was during these times that the class shared their experiences — what they 
found out, their challenges, frustrations and successes, including their plans for action. 
The seminar presentation was in phase three where they shared the PowerPoint, video 
and photographs, including posters describing the service that they had completed in 
the placement site. The data analysis using descriptive content analysis and the SoTL 
framework indicated the data sets to be presented and discussed. The research rigour of 
credibility and dependability were worked with, and the research ethics were considered 


and acted on. 
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Research Results 


Phase One 


Phase one is concerned with the expectations, informative and exploratory 
sessions at the beginning of the module and the project initiation that the pre-service 
teachers experienced. The comments written and the statements made by the pre-service 
teachers were: 


“Many things were said by other students about what we could expect in the module, the 
module is stressful, and it can be confusing etc.” 

“There is a lot of excitement and also new challenges in the module, as well.” 

“Year 2020 became so stressful, and this was due to lockdown and what is expected of me” 
“T have experienced much anxiety and stress, and this is due to how I will be able to do this 
work, now with COVID-19” 

“Confusion as to what and how I will be of service with the communities”. 


These statements indicate the emotional experiences that the pre-service teachers 
underwent. This is not surprising as Service-Learning as a concept and the pedagogy 
used to facilitate their development of understanding of the knowledge and practices is 
different to that for facilitating a Life Sciences module. Community persons, past pre- 
service teachers who had completed the module and the challenges of being expected to 
write reflections, share them, and use them for constructing and revising the thinking and 
feelings that each person experiences, is far removed from the focus on just the cognitive 
development of each person. This year with the added pressure of being at home, in their 
own communities, many rural and very far from urban areas, the search for appropriate 
placement sites was increased. 


Phase Two 


Phase two is concerned with the pre-service teacher groups’ planning for the 
interventions and the actual interventions planned. The comments made and statements 
written during the planning for the intervention included the following: 


“In planning for the groups, we struggled to find partners, especially ones that we would 
be able to work well together.” 

“T wanted to work with a partner who was not living close to me, and this gave me many 
sleepless nights” 

“Finding placement sites was very difficult and added to that when one was found the issue 
of access due to COVID-19 was a real one” 

“We spent weeks looking for placement sites” 

“We needed enough data and money to be able to contact placement sites using our phones 
and then to work out the distances for travel to the site. We will require money for transport” 
“What — how will I and my group be able to work for 25 hours in the placement site. This 
is too much to ask” 


These comments and statements were also shared during the class reflection 
sessions that we had once a week. The really purposeful and growing nature of these 
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sessions was the collective suggestions and comments shared in a safe space about the 
emotional, physical and logistical experiences that the pre-service students shared. A 
problem for one group was a solution, motivator or an idea of the possibilities that the 
groups could work with. The sharing of frustrations and problems service as a release and 
comfort in knowing that each group is not the only one having a particular experience. 

In focusing on the intervention, an analysis of these was completed, and the 
criteria used to categorise them were inductively decided on, it was the focus of each of 
the projects/programmes. The analysis of the intervention projects/programmes revealed 
the following: 


Table 1 
Categories and Number of Each of the Pre-service Teacher's Projects/Programmes 








1 Food security 2 Special Needs ciel ls 4 Environmental 5 Health related 
Related 
10 5 4 22 30 





To provide further clarity on the categorisation of the pre-service teachers’ 
projects/programmes the examples of the titles are presented below: 


Category one has 10 projects, an example, Exploring and enhancing nutrition and food 
security among grade 3 learners at a primary school; 

Category two has five projects, an example, Exploring and enhancing people living with 
cerebral palsy; 

Category three has four projects, an example, Exploring and Enhancing the Cleaner’s 
understandings of the effect of COVID-19, 

Category four has 22 projects, an example, Exploring and enhancing land pollution 
awareness, and 

Category five has 30 projects, an example, Exploring and enhancing healthfulness of 
Homeless people. 


Phase Three 


Phase three is concerned with an in-depth analysis of two of the projects. The 
visuals for each of these projects are also presented as the results: 
Project one is concerned with category five — Health related as the basis of the programme 
was about the healthy state of the child and the healthy eating practices. 
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Figure 1 
An Example of a Project in the Health Conditions Category 


ree UNIVERSITY OF ™ 
ae KWAZULU-NATAL 
+. ties INYUVESI 

wn. YAKWAZULU-NATALI 


THEMBEKILE ZWANE 218002803 
NONDUNDUZO NGUBENI 218058600 


Tittle: Exploring and educating grade R-1 learners 
about hygiene and healthy diet and environment at 
Siphesihle Primary School 





The group designed the intervention where the underlying principle is — Teach 
them until they understand the importance of hygiene and living a healthy lifestyle. 
The programme engaged learners in rhymes, music and movement in the indigenous 
language — isiZulu focusing on healthy foods and behaviours, demonstrating and acting 
on the washing of hands and the making and wearing of masks as a fashion and to be 
applauded. They also included the making and provision of healthy cooked meals (where 
they provided all the ingredients with their own money) to the children during the lunch 
break. The class group was also engaged in making and planting food crops in a school 
garden. Each child planted a plant in the garden and was responsible for taking care 
of it — watering it. The children were also educated about littering and keeping their 
environment clean, and they were engaged in a litter clean up around the school. The use 
of games was an important strategy for the learners to develop appropriate knowledge, 
skills and attitudes linked to healthy lifestyle. 

In reflecting on their experiences, the pre-service teachers presented the following 
conclusion during their seminar presentation: 


Doing Service-Learning was amazing. This gave us an opportunity to engage with different 
people and to learn and explore how quickly the minds of young children are. From this 
experience we have learnt that it is within us to do good, we learnt that when we work 
together, we can achieve a lot of things. During our time at the site, one of our mentors 
told us that they never had time to teach the learners the importance of hygiene and living 
healthy. We really appreciated and enjoyed teaching young children about the importance 
of hygiene and living healthy. 


Project two was concerned with food security, where the focus was on nutrient 
deficiency. 
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Figure 2 
An Example of a Project in the Food Security Category 
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Ngcobo (218051019) 





The pre-service teachers in this group discussed the nutrient deficiency with 
the teachers and principal at the school. They identified that the children’s diet lacked 
proteins. The group then decided that in the time that they spend at the school, 25 hours, 
they will provide meals that are rich in proteins. They purchased food items to be cooked, 
lentils, beans that are also cooked with samp, an African traditional food consisting of 
dried corn kernels. Chicken was also added to some of the cooked meals. This group also 
taught the children to plant and played games focusing on food security. 

They reflected on their experiences: 


Working with others, enabled us to develop our communication and listening skills. 
Working in groups on the sites made the experience more enjoyable. The motivation & 
encouragement we provided to each other was invaluable. We learnt a lot from Caitlin 
and Mrs Deca (the tutors) as well, sharing ideas and thoughts about their experiences in 
biology and seeing different perspectives and encouraging us to develop our own thinking 
skills and research skills. 


Conclusions and Implications 


The pre-service teacher’s experiences were varied and expansive. The 
understanding and their attitude to the module in the initial part was acceptable, as 
they had no previous experiences of research, more especially Service-Learning and 
working with a community, as part of a module expectation. Furthermore, the work 
with communities is loaded with so many aspects linked to COVID-19 safety protocols, 
and the protection of self against infection, so health, emotional, physiological and 
psychological aspects need to be considered in the work with communities for the greater 
good. During the working with communities, this experience of learning in and with their 
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community was certainly a different one for all concerned with the development of the 
mutually beneficial relationships. Mutually beneficial learning/engagement is significant 
in the emotional learning where statements made were, “working in the community is 
very amazing,” with the social learning evident from “experience was so amazing; I 
learnt the importance of spending time with other people,” and the respect “promises to 
be kept when working with a community” and attitudes of “amazing module with focus 
and honesty” and “‘work hard with honesty” were expressed. Some groups experienced 
much frustration when, “it did not end well” for them because they did not keep their 
promise of time and attendance at a placement site, and they were asked to leave. 

Service-Learning during COVID-19 was a renewed way of working and even 
with the challenges experienced by the pre-service teachers of access, finding placement 
sites or even possible ideas for interventions taking the safety and security issues into 
account. Statements like, “through this module I have learnt a lot of things, I have grown 
as a life sciences teacher, and I would also advice those who think this module is difficult 
to drop the attitude.” 

So, the enhancers are evident in the collaboration and teamwork that the pre- 
service students were engaged within and across their groups, and also with the 
placement of site persons. Working with a community for 25 hours, completing service 
that is mutually agreed upon, does entail the acceptance and role of the pre-service 
student in the place, to the achievement of the purposes identified. Since there are no 
pre-planned and clearly defined plans of actions that should be undertaken during the 
service periods, it is imperative that the pre-service teachers use their creativity and 
imagination to work with the opportunities and possibilities to design and implement a 
Service-Learning programme that is purposeful, practical, and sustainable. This entails 
the use of problem-solving strategies, which invariably are developed in that time with 
great support from the partners, lecturer, tutors, and groups for contextually relevant 
solutions to be implemented. 
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Abstract 


Primary school teachers’ science competence remains a highly relevant theoretical and 
practical problem. Although the general curricula and educational standards of Lithuanian 
general education school set quite high requirements for the process of primary school science 
education, the science competence of teachers raises reasonable concerns. It has to be stated that 
in university primary education study programmes, insufficient attention is paid to the natural 
science component. This leads to a possible lack of preparation for pre-service primary school 
teachers in science education. 

A pilot study conducted at the beginning of 2020, in which 107 students from two Lithuanian 
universities participated, showed that their scientific knowledge was poor and insufficient. 
Particularly weak was subject knowledge from the field “Nature research”, and also knowledge 
related to animate and inanimate nature. An assumption can be made that science literacy of pre- 
service primary education teachers is not sufficiently developed at university, which is one of the 
factors limiting the quality of children’s science education. There are probably two main reasons 
for this situation: insufficient science education in general education schools, and insufficient 
training of students in the field of science education for pre-service primary school teachers. 
Keywords: quantitative research, pilot research, pre-service teachers, science content knowledge, 
university students 


Introduction 


Primary science education is formed in primary school. The development of 
students’ science competence is inseparable from teachers’ science competence, an 
integral part of which are subject, didactic, and managerial abilities (Lamanauskas & 
Augiené, 2012). It is obvious that in order to understand the peculiarities of science 
education working with younger children (6-12 years), exhaustive research is needed 
and on the basis of it, science education in primary school has to be modelled, adjusted 
and developed (Lamanauskas, 2003; 2009). According to researchers, the teaching of 
science at the primary level is of world-wide concern (Lloyd, et al., 1996). 

A significant number of teachers think that they are not competent enough in 
science education (Morgan, 2012). Systematic integral science training is especially 
important for primary school teacher, they must have sufficient knowledge of biology, 
physics, chemistry, engineering and technology, ecology, etc. (Lamanauskas & 
Makarskaité, 2000). Although primary school teachers assess personal competence in 
science education quite favourably, however, they consider it insufficient (Laurukénaité 
& Makarskaité-Petkeviciené, 2003). 
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A study conducted in 2001 found that in all Lithuanian universities that train 
primary school teachers, natural science preparation of final year students is insufficient. 
In particular, they lack methodological skills, and have little knowledge of science 
education experience in foreign countries (Lamanauskas, 2001). The other research 
studies reveal similar tendencies. It is said that primary school teachers usually lack 
enough science content knowledge (Appleton, 2003). A study conducted in Estonia 
showed that student teachers’ PCK consist more pedagogical knowledge than knowledge 
of science (Timostsuk, 2015). The latter research studies also claim that primary school 
teachers are commonly known to have low self-efficacy in mathematics and science 
(Catalano et al., 2019). Teacher’s content knowledge is a critical component in teaching 
(Lewthwaite & MacIntyre, 2003), to ensure teacher’s work effectiveness (Robinson, 
2017). 

Science education in primary school is not only important but also problematic. 
The importance lies first and foremost in the fact that science education is an integral part 
of general education (Lamanauskas, 2008). The 2015 National Student Achievement 
Survey report states that “it is necessary to be concerned about the quality of science and 
social education in primary schools and to increase the percentage of fourth-graders, who 
reach the highest levels of cognitive achievement in the world” (2015..., 2015). Thus, 
the main goal of science education is students’ science literacy. The aim is for students 
to know the basic natural science concepts and conceptions, the specifics of methods, 
to develop the ability to reasonably evaluate natural science application possibilities 
and limitations. Natural science education must not only help the student get to know 
animate and inanimate nature world, but also encourage them to ask, suggest ideas, 
think critically; prove that scientific conclusions are based on clear evidence. Students 
should realise that natural sciences and their development help to create a productive 
and ecologically balanced world. Natural science learning should develop student’s 
confidence in the face of constant change, the ability to make responsible decisions and 
help to become an active citizen. 

It follows that preparation of pre-service primary teachers in the field of science 
education remains extremely relevant and significant. Science knowledge is an 
important part of their general competence. Complete and systematic research is needed 
on the basis of which primary school teachers’ preparation and further training could 
be improved. It is obvious that there is a dearth of research addressed to study how to 
improve teacher’s knowledge of science content (Verdugo et al., 2016; Martin-Raugh, 
et al., 2018). The aim of the study was to evaluate pre-service primary school teachers’ 
science content knowledge. An assumption is made that the level of knowledge is 
insufficient, and the information obtained will be useful for the improvement of teacher 
education programmes in universities and colleges. 


Research Methodology 
General Background 
Research was carried out in January-March 2020. Research type was diagnostic. 


The research was grounded on a quantitative social research paradigm, holding the 
attitude that educational diagnostics allows evaluating rather accurately the analysed 
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features. The researchers were holding an opinion that a pilot study is a small sized study 
which may be used before larger scale study of any type. 


Sample 


The study involved 107 students from two Lithuanian universities, pre-service 
primary school teachers. A convenience sampling which is one of non-probability 
sampling procedures was used. Such sample is considered sufficiently representative 
for the pilot research (Viechtbauer, et al., 2015). The sample of the study consisted of 
students, therefore the distribution of subjects according to gender is not presented. 


Table 1 
Respondents’ Distribution 

















Course N % 
The first 48 44.9 
The second 27 25.2 
The third 9 8.4 
The fourth 23 21.5 
Total 107 100.0 





Students participated in the survey voluntarily. Prior to the study, the purpose 
of the study was explained to them, and the respondents were informed about the 
confidentiality of the study. 


Instrument and Procedures 


The questionnaire used in the study consisted of 30 closed-ended questions. Each 
question had 4 possible choices, of which only one was correct. According to Jarrett et al. 
(212), multiple-choice instruments are suitable for data collection in educational surveys. 
The original instrument was used in the study (Verdugo et al., 2016). The researchers’ 
consent was received by e-mail to use the instrument, including certain application 
recommendations. The questionnaire was translated from English into Lithuanian and 
later back into English. Two researchers independently of each other ensured the validity 
of the translation. An English language expert participated in the translation validity 
assessment. 

An evaluation of the validity of the content of the research instrument was 
performed. In this regard, the general curriculum of primary science education was 
analysed, as well as other documents related to the curriculum. The basics of science 
education in Lithuanian primary schools are integrated with the basics of social education 
in the course “World cognition” This course is implemented all four years (grades 1-4 
of primary school). The content of science education includes: a) nature research; b) 
animate nature (biology); c) substances and their variations (chemistry); d) physical 
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phenomena (physics) (General Programs..., 2003). Two university lecturers, primary 
education experts, conducted an analysis of all 30 questions of the research instrument. 


It was found that they fully fit the Primary school curriculum in Lithuania. 


Table 2 


Areas of Scientific Knowledge in the Research Instrument 








Questions in : : : Fd Equivalence of the field of knowledge in 
? Main subject areas involved in primary . ‘ ‘ 
the instru- ? : : Lithuanian general education pro- 
science in Spain 
ment grammes 
Natural environment and its conservation 
1-7 Nature research 


(‘Envi " 











8-15 Biological diversity (“Life”) Animate nature (biology) 

Health education and the human body Animate nature (biology) + integrated health 
16 - 24 a P : 

(“Health”) education programme 
25 - 30 Matter and energy (‘Energy’) Substances and their variations (chemistry) 


Physical phenomena (physics) 





Thus, the analysis of the content of the curricula revealed that the issues of the 
research instrument fully fit the Primary school curriculum in Lithuania. 

The survey was conducted in the usual way, by submitting questionnaires to the 
respondents. Completed questionnaires were returned to the respondents. There were no 
damaged or incomplete questionnaires. 


Data Analysis 
For the obtained data analysis, the measures of descriptive statistics 1.e., absolute 
and relative frequencies, were applied. According to separate areas of knowledge, the 
average frequency of each was calculated as a percentage based on the correct answers. 
The average of percentages is the true average of a set of two or more percentage values. 


Research Results 


The analysed answers of the respondents are presented in table 3. Not only the 
results of the correct answers are presented, but also the incorrect answers. 
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Table 3 


Students’ Science Content Knowledge 





Question 


Correct answers, N (%) 


Incorrect answers, % 





1. What is the movement of the 
Eartharound the Sun called? 


Revolution, 24 (22.4) 


Rotation, 64.5 
Precession, 12.1 
Circumference, 0.9 





2. How long does it take the Moon 


28 days, 51 (47.7) 


365 days, 26.2 





to orbit the Earth? 21 days, 13.1 
7 days, 13.1 
3. Climate is... The meteorological The meteorological conditions produced 


features occurring in a 
specific region over a long 
period of time, 50 (46.7) 


by changes in 

atmospheric pressure, 39.3 

The set of atmospheric phenomena 
occurring in a 

specific time and place, 13.1 

The condition of the atmosphere for a 
short period of time (less than 30 days), 


0.9 





4. Which of the following state- 
ments is correct? 


Rocks are made of miner- 
als, 56 (52.3) 


Minerals, like marble and granite, are 
very useful forconstruction, 32.7 
Minerals are made of rocks, 8.4 
Rocks have a low value for human 
beings due to their 

limited applications, 6.5 





5. What kind of device is used for 
measuring wind speed? 


Anemometer, 21 (19.6) 


Barometer, 54.2 
Weathercock, 25.2 
Pluviometer, 0.9 





6. What is the name given to the 
groups formed by gas, interstellar 
dust and thousands or millions of 
stars? 


7. What part of a volcano is the 
conduit? 


Galaxies, 65 (60.7) 


Pipe by which magma 
flows up, 76 (71.0) 


Constellations, 23.4 
Nebulas, 9.3 
Planetary systems, 6.5 


Orifice connecting volcano to the exterior, 
15.0 

Rocks and solid material formed when 
lava gets cold, 9.3 

Underground pool where magma resides 
temporary, 4.7 





8. Which of the next animal groups 
are invertebrates? 


Insects, 64 (59.8) 


Amphibians, 30.8 
Fishes, 9.3 





9. Living things can be arranged in 
levels of complexity. When some 
tissues co-work to perform the 
same function, what do they form? 


Organs, 36 (33.6) 


Apparatus, 50.5 
Interwoven systems, 9.3 
Molecules, 6.5 
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10. What is the vital function of 
flowers in plants? 


Reproduction, 77 (72.0) 


Respiration, 15.9 
Interaction, 10.3 
Nutrition, 1.9 





11. In addition to water and sun, 
what gas do plants need to carry 
out the photosynthesis? 


COQ2 (carbon dioxide), 58 
(54.2) 


O2 (oxygen), 39.3 
CO (carbon monoxide), 4.7 
N2 (nitrogen), 1.9 





12. What is the name given to 
the leaves that form the calyx of 
a flower 


Sepals, 57 (53.3) 


Petals, 27.1 
Stamens, 15.0 
Corolla, 4.7 





13. What is the function of phloem 
in aplant? 


Carrying elaborated sap 
from the leaves to the rest 
ofthe plant, 21 (19.6) 


Carrying raw sap from the roots to the 
rest of the plant, 47.7 

Absorbing water and mineral salts from 
the soil, 26.2 

Absorbing rays from solar light, 6.5 





14. What biological kingdom do 
viruses belong to? 


No kingdom, 45 (42.1) 


Protista, 24.3 
Monera, 22.4 
Fungi, 11.2 





15. Which of the following state- 
mentsabout differences between 
plant cells and animal cells is 
correct 


16. Where is the exchange be- 
tween O2and CO2 produced in the 
respiratory system? 


Plant cells contain orga- 
nelles called chloroplasts 
that animal cells do not, 
79 (73.8) 


Pulmonary alveoli, 69 
(64.5) 


Plant cells do not have cellular mem- 
brane and animatells do, 12.1 

Animal cells have mitochondria and plant 
cells do not, 10.3 

There are no differences. All living organ- 
isms are made up of the same cells, 3.7 


Bronchi, 12.1 
Trachea, 12.1 
Bronchioles, 11.2 





17. Which organ is responsible for 
filtering the blood to remove waste 
substances? 


Kidneys, 53 (49.5) 


Liver, 43.0 
Urinary bladder, 7.5 





18. Which blood component has 
thefunction of carrying oxygen? 


Red cells, 44 (41.1) 


White cells, 22.4 
Plasma, 18.7 
Platelets, 17.8 





19. What is the name given to the 
cellsresulting from fertilization? 


Zygote, 73 (68.2) 


Ovule, 17.8 
Gamete, 10.3 
Spermatozoon, 3.7 





20. Most nutrients pass into the 
blood in... 


The small intestine, 51 
(47.7) 


The stomach, 29.9 
The large intestine, 16.8 
The oesophagus, 5.6 





21. In the excretory system, what 
ductconnects to the outside of the 
body? 


Urethra, 15 (14.0) 


Ureter, 44.9 
Anus, 35.5 
Urinary bladder, 5.6 





22. Which organs form the central 
nervous system? 


The brain and the spinal 
cord, 79 (73.8) 


The cerebrum, the cerebellum and the 
medulla oblongata, 14.0 

The sensory and motor nerves, 7.5 
The cerebrum and the cerebellum, 4.7 
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23. Which organ produces bile? 


Liver, 18 (16.8) 


Pancreas, 71.0 
Stomach, 10.3 
Small intestine, 1.9 





24. What nutritional substances, 
present in cereals and legumes, 
provideus with energy? 


Carbohydrates, 69 (64.5) 


Proteins, 31.8 
Lipids, 2.8 
Mineral salts, 0.9 





25. Which of the following 
methods would you use in order to 
separate a solid from a liquid ina 
heterogeneous mixture? 


Filtration, 60 (56.1) 


Distillation, 26.2 
Crystallisation, 15.0 
Magnetisation, 2.8 





26. What is the bending of light 
rayscalled when they pass from a 
fast medium to a slower one? 


Refraction, 66 (61.7) 


Diffraction, 20.6 
Reflexion, 11.2 
Attenuation, 6.5 





27. Which of the following state- 
mentsabout renewable energy 
sources is correct? 


They are considered inex- 


haustible energy sources, 
49 (45.8) 


Thermal solar energy is an energy 
source that generates 

electricity by means of photo-voltaic solar 
panels, 39.3 

They are at present time the most com- 
mon energy sources, 15.0 





28. What kinds of changes modify 
the composition of matter? 


Chemical changes, 76 
(71) 


None. Matter does not change, 12.1 
Physical changes, 11.2 
Biological changes, 5.6 





29. What characteristic of sound Pitch, 82 (76.6) Timber, 13.1 
tells us the difference between a Intensity, 7.5 
high sound and a low sound? Volume, 2.8 
30. What colour would be obtained — White, 40 (37.4) Black, 42.1 
if all the colours of the rainbow Red, 17.8 
were mixed? Yellow, 2.8 





Analysing the respondents’ answers to the questions in the first area (1 - 7 
questions), it can be seen that the percentage only to three out of seven correct answers 
exceeded 50. The worst result was the answer to the question “What kind of device is 
used for measuring wind speed?”. Only 19.6% of respondents gave the correct answer. 
The largest share (54.2%) answered that it was “a Barometer”, which is designed to 


measure atmospheric pressure. 


Assessing the respondents’ knowledge according to the second area (questions 
8 - 15), the results were slightly better. The percentage of correct answers to 5 questions 
out of 8 exceeded 50. The worst knowledge was relatively related to animal and plant 
anatomy and morphology. Only 19.6% of respondents answered correctly to the question 
“What is the function of phloem in a plant?”. Nearly a quarter of respondents attributed 


the viruses to the kingdoms of Protista and Monera. 


Analysing students’ answers to the questions in the third area (questions 16 - 24), 
it can be seen that their knowledge of human body and health is insufficient. Only the 
percentage of correct answers to 4 out of 9 questions exceeded 50. The worst answers 
were to the question “Which organ produces bile?”. Only 16.8% of students gave the 
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correct answer. The largest share of respondents (71.0%) indicated that bile was produced 
at the pancreas. The same also can be said about the question “In the excretory system, 
what duct connects to the outside of the body?”. Only 14.0% of respondents gave the 
correct answer. 

Students’ knowledge in the fourth area (questions 25 - 30) was the best compared 
to other areas. The percentage of correct answers to 4 out of 6 questions exceeded 50. 
The worst results were observed in the answer to the question “What colour would 
be obtained if all the colours of the rainbow were mixed?”. Only 37.4% of students 
answered this question correctly. Similarly, 1.e., as insufficient can be evaluated students’ 
knowledge about renewable energy sources. Only 45.8 of respondents gave the correct 
answer. 

The fourth table summarises the results according to the correct answers provided. 


Table 4 
The averaged percentage of correct answers by content knowledge areas 








The first area The second area The third area The fourth area 
(1-7) (8-15) (16-24) (25-30) 
45.8 51.0 48.9 58.1 





It can be clearly seen that students’ science knowledge is poor. The best results 
are demonstrated in the fourth area, the worst — in the first. On the other hand, no final 
generalisations can be made, but the overall situation is quite clear. In each of the areas, 
there are questions that were answered correctly by the majority of respondents (the 
percentage of the correct answers exceeds 70). On the other hand, it is becoming clear 
that there are clear gaps of scientific knowledge. 


Conclusions and Implications 


The main research aim was to assess science content knowledge in pre-service 
primary school teachers. The results show that natural science knowledge is clearly 
poor and insufficient. The analysis of answers to some questions highlighted possible 
and serious pre-service teachers' misconceptions in science content. It can be stated that 
the best expressed area is “Substances and their variations”, and the worst is “Natural 
research” area. 

Although the study has several limitations (small sample, no detailed analysis 
of wrong answers), the results still indicate the need to review primary school teacher 
preparation programmes. Preliminary analysis of study programmes shows that little 
attention is paid to the natural science component (natural science competence formation). 

The development of specific subject knowledge (how to use it in specific 
educational situations) becomes especially important. In Lithuania pedagogical skills 
are separated from subject knowledge, even in competence development events, the 
formation of pedagogical skills often takes place without a specific content of the 
subject. Such access just limits the management of subject knowledge, the transfer in 
specific situations. A similar situation is observed in the training process of primary 


school teachers. 
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Abstract 


This study describes a research focused on primary teachers’ evaluation of their students’ 
digital literacy as a basic competence in the structure of natural science literacy of research 
and comprehension competence. With the term, primary teachers we mean teachers of first three 
grades of compulsory school. The comparison of basic computer skills between Generation Z and 
Generation Alpha revealed, contrary to expectations, a very small progress. The consequence 
of this circumstance are the problems associated with the implementation of natural science 
curricular goals in digital environments of remote teaching in school closure period. A 
questionnaire given to 176 primary teachers, revealed which digital learning environments could 
be chosen for science lessons and which curricular goals could/could not be achieved with this 
level of Generation Alpha students’ digital literacy competence. 

The results showed that the digital environment is more problematic and unfavorable for 
Generation Alpha in the field of natural science, as teachers showed a higher frequency of 
avoiding natural science goals than social science content. 

Keywords: distance learning environment, generation Alpha, natural science digital literacy 
competence, natural science education 


Introduction 


Since the beginning of the third millennium, it had been clear that learning 
environments at all levels of the educational system would (and must) change, as a 
consequence of the digitalization process in professional and personal lives and as a 
consequence of the changes that this shift in communication from analogue to digital 
had made in the human brain. But no one had imagined that the shift from traditional to 
digital would happen in an instant, for all levels of education, for all students, and for all 
teachers — as it did with the closure due to COVID-—19 lockdown all around the world. 

Well, this change in learning environments took place in different contexts: There 
were countries (regions) that were technologically prepared. Their internet connections 
could handle multiple users per time, their students were equipped with computers and 
tablets, their teachers were digitally competent to teach in digital environments and, 
the students were already digitally competent to participate in the process of distance 
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learning in digital learning environments. It is assumed that all these factors occur 
simultaneously, and they usually do, but in the case of primary level of education, the 
last-mentioned factor — students’ digital literacy — is a constant problem even in digitally 
advanced technological and digital learning environments. 

In the context of natural science didactics in digital learning environments, the 
crucial question of primary level natural science education — besides the basic literacy 
problem — is students’ natural science digital literacy competence. Previous research on 
students’ digital competence for learning in digital environments points in two directions: 
Previous studies assumed that “digital natives” already enter school digital competent. 
Even more: Their digital competence is so far developed that they have nothing more to 
learn from their teachers, “digital immigrants” (Prensky, 2011; Spiro, 2004). Later studies 
(Cornu, 2011; Sadler, 2011; DeStefano&LeFevre, 2007; Kordigel AberSek et. al., 2015), 
which measured the relationship between learning in digital environments and learning 
outcomes, noticed a remarkable difference between students with relatively high level of 
digital pre-knowledge and those with weak pre-knowledge. Meanwhile the generation 
changed. The primary grade students do not belong to the millennium generation, or, 
as they were called: Generation Y, anymore. They belong to Generation Alpha — they 
are children of Millennials and younger siblings of Generation Z. Generation Alpha is 
a generation born in (or after) 2010, the year the iPad was launched, and Instagram 
was created. Not only were their parents constant users of smart devices themselves 
(unlike the parents of Millennials, who struggled with new technologies), they used 
smart devices as pacifiers, entertainers, and educators, as screens were put in front of 
their children at the youngest possible age (McCrindle & Fell, 2020a). The consequence 
of a different digital environment in early childhood could lead to significantly higher 
level of (at least basic) digital competence at the age when children — Generation Alpha 
— enter school, which would enable them to successfully participate in digital learning 
environments (McCrindle & Fell, 2020b). This optimistic speculation usually overlooks 
what is expected of learning environments in the 21“ century (DuMond & Instance, 
2010). For natural science teaching/learning, the learning environment should provide 
the opportunity for a social-constructivist approach to learning, where learning/teaching 
takes place in the interaction between learners and their contextual situation, with learners 
actively constructing their knowledge and skills. Learning in such learning environments 
is/should be self-regulated, with active use of learning strategies, it should be situated in 
context and not abstracted from the environment, and it should be collaborative - not a 
solo activity (De Corte, 2010). 


The Aim of the Study 


The first aim of the research was to find out if and to what extent the digital literacy 
of Generation Alpha is advanced in comparison to the digital literacy of Generation Z. 
The second aim was to find out whether the digital literacy of primary students, age 6 — 8, 
reaches the level that can be used in the context of natural science digital competence for 
successful participation in social-constructivist based natural science class. 

To find the answer to this central research question, the following research 


questions were formed: 
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1. According to the teachers' assessment, how high is the level of primary 
students’ basic computer literacy — the basic competence of natural science 
literacy of research and comprehension competence? 

2. What digital learning environments did teachers use for science teaching 
during school closure in Slovenia? 

3. How many natural science curricular goals were reached in these digital 
learning environments? 

4. What natural science curricular goals were not reached in the digital learning 
environments and what are the reasons why teachers did not choose them 
during distance learning in the digital learning environments? 


Research Methodology 


The research background for establishing the difference/progress of digital literacy 
of Generation Alpha compared to the previous generation (Generation Z) was the study 
conducted in Slovenia in spring 2015, which examined compulsory and secondary school 
natural science teachers’ assessment of their students’ new natural science literacies of 
online research and comprehension competence. 


Sample Selection 


Asample were primary school teachers in Slovenia. Why in Slovenia? As mentioned 
above, four main factors influence students' performance in digital environments: 

¢ technological factors (accessibility and quality) of internet connections, 

¢ — students' equipment with computers and tablets, 

* teachers’ digital competence to teach in digital environments and 

¢ the fact that students were already digitally competent to participate in the 

process of distance learning in digital learning environments. 

These four factors usually occur together, but not in Slovenia: In Slovenia both 
technology factors were solved very quickly, teachers were prepared to use digital 
learning environments in many projects. 

Consequently, Slovenian primary school teachers are a reliable sample to find 
answers to the listed research questions. Our sample consisted of two sets of data. 135 
teachers assessed their students' basic online skills, while 183 teachers responded to 
the questions about teaching in digital learning environments. The sample was random 
selected. Participants were not asked to give the sociodemographic data, because the 
research was not focused in interconnectedness between teachers’ gender/the length of 
their teaching experience and the natural science teaching practice in distance teaching 
period, but in their evaluation of students’ digital competence and in their teaching 
practice, connected with natural science curricular goals. 


Ethical Procedures 


Participants’ approval was obtained in research in line with the voluntary 
principle. All participants were informed about the purpose of the research. At all stages, 
the identities of the participants were kept, and the codes, given to the participants were 
used when quoting raw data texts. 
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Instrument and Procedures 


The first set of data was collected using a 5-point Likert scale questionnaire, 
consisted of 21 items. The instrument was adapted from the TICA checklist, developed 
as part of the Teaching Internet Comprehension Skills to Adolescents project, which 
focused on learning skills, essential for online reading comprehension (Leu et al., 2008). 
The original TICA online reading comprehension checklist included items from five areas 
necessary for online reading comprehension: understanding and developing questions, 
locating information, critically evaluating information, synthesizing the information, and 
communicating the information. The adaptation used for our study focused on computer 
basics (Appendix A in the original survey), and the number of items was reduced — from 
29 to 21 in the computer basics checklist. 

The second set of data was collected through the online questionnaire Learning 
environment for the first three grades in the time of remote teaching, which contained 52 
questions regarding teachers’ experiences after COVID—19 school closure (11 weeks in 
Slovenia). 


Data Analysis 


Quantitative data from primary school teachers were collected. After verifying 
that the data were free of errors, quantitative analyzes were conducted and analyzed 
according to the following stages, or by encoding, defining, and organizing the data and 
interpreting the results. For the statistical processing of the data an IBM SPSS program 
was used. For basic statistical interpretation of the results, number, structural percentage, 
mean and standard deviation of the data were used. 


Research Results 


The findings ascertained from this research are presented in two parts. The first 
part presents data of primary school teachers’ judgement of their students’ computer 
basic skills, their web searching basics and general navigation basics, skills that are the 
basement of digital competence, needed for participation in digital learning environments. 
The second part presents data about natural science education in the period of school 
lock down, implementation of natural science curricular goals and reasons, why certain 
natural science curricular goals were not chosen for didactic units in digital learning 
environments. 
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Table 1 
Primary School Teachers’ Ratings of Their Students’ Basic Online Skills 



















































































Basic online skills Generation Z (2015) | Generation Alpha (2021) 
M SD M SD 
1. Computer Basics 
Turn a computer on/off 441 1.07 4.6 0.89 
Use the mouse/track pad 4.35 0.91 4.29 0.83 
Follow classroom and school rules for computer use 3.47 1.14 3.70 0.91 
te ah and files using icons and/or the Start 288 0.82 332 0.91 
Create/open a new folder/file 1.94 0.95 2.05 0.89 
Launch a word processor 2.06 0.84 2.16 1.00 
Open a word processing file 1.88 0.97 2,15 1.02 
Type a short entry in a word processing file 2.00 0.83 1.89 0.95 
Copy text 1.76 1.23 1.97 0.95 
Cut text 2.24 1.04 2.35 1.11 
Past text 1.82 1.01 1.95 0.90 
Name a word processing file and save it 1.71 0.97 1.90 0.99 
Open a new window 2.24 1,13 2.36 1.11 
Open a new tab 1.65 0.87 1.59 0.79 
2. Web Searching Basic 
Locate and open a search engine 2.00 0,85 3.35 1.27 
ae words in the correct location of a search 271 0.91 3.06 4.24 
Use the refresh button 1.94 1.14 2.25 1.10 
Use the “BACK” and “FORWARD” buttons 2.65 0.82 2.84 1.12 
3. General Navigation Basic 
Maximize/minimize windows 2.18 0.7 248 1.08 
Open and quit applications 2.76 0.81 3.26 1.18 
Toggle between windows 2.00 0.89 2.58 1.03 





Comparison of the results shows that basic computer competence in Generation 
Alpha developed less than expected in the context of the digital stimulating environment 
from the earliest age. Over the last 6 years, the results have changed only slightly. 
Generation Alpha enters the school hardly computer literate. Almost all of them can turn 
the computer on and off and use the mouse/trackpad, which they had probably learned 
while using digital devices for play/fun and perhaps socializing, but a large majority 
of them do not have other 21 basic computer skills that they would have needed to 
participate in the educational process in digital learning environments. 
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Table 2 
Learning Environments for Teaching Science in the Distance-Learning Period 





allude Social Phone 
Videoconference Email learning Other* Other 
‘ networks calls 
environment 


















































f 2 32 66 149 79 104 51 

f% 1.14 18.18 37.50 84.66 44.89 59.08 80.95 
f 3 14 4 6 23 13 1 

: f% 1.70 7.95 2.27 3.41 13.07 7.39 1.59 
f 0 7 1 1 22 17 1 

: f% 0.00 3.98 0.57 0.57 12.50 9.66 1.59 
f 13 35 6 9 26 24 3 

: f% 7.39 19.88 3.41 5.11 14.77 13.64 4.76 
f 61 30 6 5 17 11 3 

° f% 34.66 17.05 3.41 2.84 9.66 6.25 4.76 
f 97 58 93 6 9 E 4 

° f% 55.11 32.96 52.84 3.41 5.11 3.98 6.35 

176 176 176 176 176 176 176 

i f% 100.00 100.00 100.00 100,00 100.00 100.00 100.00 

SD 5.4 44 3.9 1.5 2.5 2.1 17 

Mean 0.9 1.9 24 1.3 1.6 1.6 1.6 





Note: Other* — Mentimeter, BookCreator, Padlet, Plickers, Quizlet, Kahoot, Thinglink ...; 1 — Never, 
2 — Once a month, 3 — Once every two weeks; 4 — Once a week, 5 — Up to 3 times a week, 6 — Daily 


Students' limited basic computer skills did not prevent teachers from using 
digital learning environments under distance learning conditions during the COVID-19 
closure. The results in Table 2 show that more than half of them used video conferencing 
(Zoom, Teams, Meet, etc.) almost daily, a similar percentage used Moodle, Mahara, 
0386, Seesaw, Google classroom, classroom e-assistant Xooltime (online learning 
environment), only one third used email connection. These results can be explained in 
the context of parents’ complaints about being overloaded with teachers’ expectations for 
participation in primary students' homeschooling during the COVID—19 school closure. 
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Table 3 
Science Curricular Goals Reached during the Remote Teaching Period 


























f f% 

One fifth of curricular goals and less 2 1.30 

A quarter of curricular goals 4 2.60 
One third of the curricular goals 13 8.44 
Half of the curricular goals 21 13.64 
Two-thirds of the curricular goals 19 12.34 
Three-quarters of the curricular goals 41 26.62 
Almost all or all curricular goals 54 35.06 
Total 154 100.00 





Table 3 shows that only a little over one-third of the teachers managed to reach 
all of the science curricular goals according to the annual plan, they prepared before the 
October closure. A quarter (25, 98 %) of the teachers admitted that they were only able 
to achieve half or even less of the science curricular goals. These results urge us to seek 
the answer to the question presented in Table 4: What science curricular goals teachers 
decided to teach/not to teach in digital learning environments. 


Table 4 
Curricular Goals, for which Primary Teachers Decided that They Are/Are Not 
Achievable in Digital Learning Environments 





Achievable goals Unachievable goals 
Goal 1 Goal 2 Goal 3 Goal 1 Goal 2 Goal3 Goal4  Goal5 
f f%# fF f% ff f% fF Ff% fF Fh F Fo F FH FF F% 











Social 
sciences 


Biology 10 2857 4 19.05 5 26.32 4 2500 4 3333 1 16.67 1 50.00 2 100.0 
Physics 11 3143 4 19.05 6 31.57 6 3750 3 25.00 5 83.33 0 0.00 0 0.00 
Total 35 100.0 21 100.0 19 100.0 16 100.0 12 100.0 100.0 2 100.0 2 100.0 


14 40.00 13 61.90 8 4211 6 3750 5 41.67 0 0.00 1 50.00 0 0.00 











fo>) 





The results in Table 4 show that most of the achievable goals are from the field 
of social sciences, while several goals for which primary school teachers decided they 
are not achievable in digital learning environments derives from the field of physics and 
biology. These results initiated the next research question: which were the reasons for 
teachers, not to choose the curricular goals from the fields of natural sciences to engage 
with in the digital environment. 
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Table 5 
Reasons for Not Selecting Natural Science Curricular Goals in the Digital Learning 
Environment 
1 2 3 4 5 6 7 Total 

f f% Ff £%H F F% F F% Ff FH FF Fo F FH $F F% 
Goal1 2 435 1 217 9 19.57 8.70 20 43.47 15.22 3 652 46 100.00 
Goal2 2 9.09 1 455 2 13.64 2 9.09 22 100.00 





4 
9.09 2 9.09 10 45.45 
Goal3 1 9.09 0 0.00 1 9.09 0 





Goal4 1 25.00 0 0.00 1 2500 1 0.00 2 50.00 0.00 0 0.00 4 100.00 





7 
3 
0.00 5 4546 3 27.27 1 9.09 11 100.00 
0 
0 





Goald 0 0.00 0 000 0 000 1 3333 2 6667 0.00 0 0.00 3. 100.00 
Note: | — Students were not present, 2 — The curriculum 1s overloaded, 3 — The goal 1s too demanding, 
4 —No digital learning resources available, 5 — Distance learning requires a different didactic approach, 
6 — Limited students' digital competence, 7 — My limited digital competences. 


In the field of natural sciences (goals 1, 2 and 3), teachers mostly decided not to 
teach in digital learning environments because distance learning requires a different 
didactic approach, followed by the goal is too demanding and limited student’ digital 
competence (Table 5). 


Discussion 


The results from our research are consistent with previous results, gained in a 
meta-analysis by Delgado and his team (Delgado et al., 2018). As we, they expected 
newer generations to achieve better learning outcomes in digital environments than older 
ones. In their meta-analysis, which included studies from 2000 to 2017 with a sample of 
171.055 participants, they examined, whether the publication date of the study »reveals 
a decreasing advantage of paper in recent years due to greater exposure to technology 
than in earlier years« (p.7). Contrary to expectations, the meta-analysis showed that 
earlier (and longer) exposure to digital media has a diametric effect (Duncan et al., 2015; 
Pfost et al., 2013): Not only were there no differences between age groups across the 
years, but the results also obtained later show poorer reading and learning outcomes 
in digital environments. This is consistent with our findings showing that, contrary to 
expectations, primary students — Generation Alpha — do not have sufficiently developed 
digital competence to participate effectively in digital learning environments — which 
presents a major challenge for primary teachers, as our research has shown, how to 
perform the natural science class. 


Conclusions 
In conclusion, a call is needed for researchers, policy makers, and educational 


experts to develop methods that support effective digitally based natural science teaching 
and learning in digital learning environments for very young Generation Alpha students, 


https://doi.org/10.33225/BalticSTE/2021.105 | | 











| Proceedings of the 4" International Baltic Symposium on Science and Technology Education, BalticSTE2021 


113 


who have very limited digital literacy skills in addition to their still emerging general 
literacy skills. 
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Abstract 


The spread of COVID-19 has caused a high demand for online learning or self-learning at home. 
We focus on evaluating which media is most suitable for self-learning for the third graders. For 
effective assessment of children's learning, the study adopted AEIOU: Awareness; Enjoyment; 
Interest; Opinion formation; and Understanding as the evaluation framework. 

In this research, three types of media were implemented: animation, educational video, and 
science comics. Total 145 third graders were divided into three groups: animation group, 
educational video group, and science comics group, to learn the concept of heat transfer through 
one kind of media respectively. The results revealed that students had positive feedback on the 
three dimensions of Awareness, Enjoyment, and Interest. And all three media improved students' 
understanding of heat transfer without a difference. While in the dimension of Opinion formation, 
the students had less life experience and pre-knowledge, their performances were relatively low. 
In summary, the three types of media showed the effectiveness of self-learning for the third 
graders. Especially the animations proved to be the most suitable for the third-grade children to 
conduct self-learning. 

Keywords: science communication, AEIOU, heat transfer, Media 


Introduction 
Background and Research Purposes 


Since the spread of COVID-19 in 2019, countries have begun to adopt different 
lockdown measures as the epidemic has become more and more severe. This includes 
decisions that affect education such as closing schools, suspension of schools, suspension 
of classes, and so on. Antonio Guterres (2020), the ninth secretary-general of the United 
Nations, also pointed out: As of mid-July 2020, more than 1| billion students have been 
affected by the suspension, which results in at least 40 million children not able to attend 
preschool. Such learning crisis may cause "generational catastrophe". Therefore, the 
needs of using media for online self-learning and self-learning at home have arisen. 
Regarding online or home-based self-learning, it is necessary to use media to deliver 
the information. This study aimed to investigate which type of media can convey the 
concept of heat transfer to elementary school students most efficiently? Which types 
of media are elementary school students more interested in? Previous studies reported 
that the use of media should also consider how to communicate efficiently. The function 
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of media is not only to deliver the scientific information, but also to shape scientific 
information (Dimopoulos & Koulaidis, 2002). If media can correctly disseminate 
scientific information, it can improve the public's judgments and enable them to illustrate 
ideas clearly when participating in the discussion of scientific issues. In addition, it 
encourages the public to actively participate in the formulation of science and technology 
policies (Chin & Chen, 2007; Norris & Phillips, 2003). In order to correctly evaluate the 
effectiveness and impacts of different types of media used, it is necessary to first explore 
the definition of science communication, to understand which aspects can be used to 
evaluate the purpose of this research and develop evaluation instruments. 


Science Communication 


As the well-known Bodmer Report (The Royal Society, 1985) highlighted the 
issue of the insufficient scientific knowledge in public, the public began to realize that it 
might affect the development of science, and this resulted in several waves of research and 
discussion on science communication. Since the 1990s, the main model for research and 
practice of science communication has changed from the deficit models to the engagement 
models (Wynne, 1992; Sturgis & Allum, 2004). The deficit models emphasized the 
one-way knowledge transmission from teachers to students, stating that if the public 
does not have sufficient scientific knowledge, it will indirectly affect the support for 
science. On the other hand, the engagement models advocated that the decision-making 
of public issues be participated by the public in two-way communication, as opposed 
to the past decision-making style dominated by the experts (Hung, 2017). From this 
point of view, related scholars in science communication believe that the public must be 
able to make more meaningful decisions when facing science-related issues in society. 
Therefore, the public should have more and more extensive conversations with teachers 
or scientists, not just receiving knowledge (Baram-Tsabari & Osborne, 2015). To be able 
to effectively evaluate the effectiveness of a two-way communication, the definition of 
science communication must first be clearly stated. 

Burns et al. (2003) stated that the boundary between the definition of science 
communication and the terms used in the field of scientific literacy was blurred. In 
view of this, Burns et al. (2003) pointed out the ambiguity between the definition of 
science communication and the terms used in the field of scientific literacy. In view of 
that, Burns et al. (2003) compiled a list of elements used in a two-way communication 
model (e.g., engagement model), for example: public awareness of science (PAS), public 
understanding of science (PUS), scientific culture (SC), or scientific literacy (SL), etc., 
and further listed the definitions that are suitable for the science communication. When 
there is a definition of the science communication, it cannot only promote science, but 
also serve as a basis for evaluating the effectiveness of science communication. 


Theoretical Framework 


Burns et al. (2003) defined science communication as “the use of appropriate skills, 
media, activities, and dialogue to produce one or more personal responses to science” 
(p.191). These responses are classified into five categories (the AEIOU vowel analogy): 
Awareness, Enjoyment, Interest, Opinion formation, and Understanding. AEIOU can be 
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used as the common goal of science communication and science education. It also provides 
a framework for this research to evaluate the effectiveness of science communication, 
which can be used to analyse the public's response to science communication activities 
or experiences. The details of each category are described below. 

(1) Awareness: For science, it provides a basis for knowledge, broadens thinking, 
and creates opportunities for individuals to participate in scientific activities 
(Burns et al., 2003). Awareness of science is an important consideration for the 
public to participate in science communication activities. This study defines 
awareness as being aware of the importance of science and technology and 
being able to understand nature of science. 

(2) Enjoyment: Enjoyment may evoke positive feelings and attitudes of 
participants, so that participants are more likely to produce subsequent and in- 
depth scientific activities (Stocklmayer & Gilbert, 2002). This research regards 
enjoyment as the pleasant life experience brought by today's technology. 

(3) Interest!|Krapp et al. (1992) stated that innovation and appropriate scientific 
communication activities can inspire the personal interest of participants. This 
activity can also stimulate participants' interest in the situation and further 
enhance participants' recall and understanding of the event. This research 
regards interest as the tendency and intention to participate in scientific issues, 
events, or activities. 

(4) Opinion formation: Opinions are closely related to knowledge, beliefs, and 
emotional responses, and are deeply influenced by them (Crawley & Koballa, 
1994). Therefore, this part of the research is not easy to implement. The most 
effective science communication one can produce is for participants to reflect 
on, form, reform or confirm their attitudes towards science and society (Burns 
et al., 2003). In this research, the formation of opinions is defined as opinions 
or viewpoints formed in the reflection of the public to support their arguments 
on certain scientific issues. 

(5) Understanding: Understanding of science includes comprehension of scientific 
content, processes, and social factors (Burns et al. 2003). It emphasizes the 
application and meaning of science. This research defines this understanding 
as the understanding of the acquired scientific knowledge. 

Science communication is committed to enhancing the awareness of science, 

scientific understanding, scientific literacy, and scientific culture through the AEIOU 
responses established by participants (Burns et al., 2003). 


Research Methodology 
Background 


This research adopted a quasi-experimental design, the purpose was to explore 
the impacts and effectiveness of using different media to illustrate the concept of heat 
transfer among the third-grade students in elementary school. The participants of this 
study were six classes of the third-grade students, later divided into three groups: two 
classes were animation group, two classes were educational video group, and two classes 
were science comic group. Each group was used for three standard class periods which 
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were 120 minutes in total. Based on AEIOU as the evaluation framework, each group 
conducted the pre-test of Understanding test, and then watched the media of each group. 
The media materials included animations, educational videos, and science comics that 
were all about the concept of heat transfer. Afterward, the groups performed the post-test 
of Understanding test; subsequently, each group had to watch or read the other media. 
After watching or reading all three media, students would take the AEIO questionnaire. 
Finally, statistical analysis on the quantitative results of the AEIU was used, for the open- 
ended responses from O need to be coded before analysis. 


Participants 


A total of six classes of third-grade students in an elementary school in Taiwan 
were chosen as participants (V = 145) with an average age of 9 years old. This research 
used a cluster sampling. The two classes were treated as one group, and six classes were 
divided into three groups: animation group, educational video group, and science comic 
group. There were 49 people in the animation group, 48 people in the educational video 
group, and 48 people in the science comic group. Before the experiment teaching, all 
the participants had not been exposed to the related concepts of heat transfer. Because of 
the small scale of samples, the inference of the research results was limited, and it was 
only suitable for the students who infer to the sample of this study and situations similar 
to this research. To infer this research results to other groups or educational fields, the 
applicability needs to be evaluated carefully. 

Introduction of media used in the experiment 

This study used three different media; the description of these media is as follows: 

(1) Animation: In 1980, the International Association of Film Animation 
(ASIFA) defined animation as " in addition to real actions or methods, various 
techniques are used to artificially create dynamic images " (Liu, 2009). The 
animation materials used in this study were from Junyi Academy website, 
which is a non-profit educational organization in Taiwan that uses an online 
learning model. This website focuses on providing equal and high-quality 
educational resources for every child, and through technological instruments 
to help teachers and parents to teach in accordance with learners’ aptitude. 
Thus, let children develop the habit of self-learning and become life-long 
learners (Junyi, 2021). Since the animation playback will be paused during 
the question time, it will continue to play only after the question is answered, 
which increases the opportunities for online self-learning and interaction. 

(2) Educational Video: The educational video provides both sound and image 
elements at the same time. Learners can use the continuous sound and images 
to more easily understand the details that originally required a lot of text 
description. The educational video does not have an interactive design, and 
the learners receive the video information unidirectionally, and the learning 
process is relatively boring (Huang, 2014). The educational videos used in 
this research are specially filmed with textbook content. The advantage of 
the educational videos is a direct presentation of a realistic situation. With 
narration, text, and simple symbols (such as arrows), the teaching content 
can be presented simply and clearly, and the learners will be more deeply 
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(3) Science Comic: Xiao (2002) has made a simple analysis of the composition 
of comics, dividing the elements of comics into four parts: "Image", "text", 
"frame", and "story". The Science Comics used in this research refers to comics 
that contain scientific elements, and " The King of Science Experiments (10)- 
The Flow of Heat" was used as the reading material for the science comics 
group. 

The King of Science Experiments is published by San Cai Publishing House. It 
was originally a South Korean comic and has now been translated into Chinese. This set 
of books takes "competition for experimentation" as the main plot, and the protagonist 
participates in the experiment competition and brings out learning knowledge. In 
addition to comics throughout the book, there are also collocation articles to tell scientific 
knowledge. However, the scientific knowledge content of this book is relatively less. 
The whole book contains 200 pages of text, 183 pages of comics, 17 pages of scientific 
articles, and 15 pages of knowledge related to heat transfer. 


Table 1 
The Media Used in the Experiment 














Media type Source Heat transfer conception Time 
heat conduction 
Animation Junyi academy website — heat convection 10 minutes 58 seconds 
heat radiation 
Educational Online resources heat conduction 
; produced by textbook heat convection 10 minutes 36 seconds 
Video : ae 
publishers heat radiation 
heat conduction 
F ; The King of Science heat convection : 
scien GOMis Experiments (10) (Note: Another heat radiation Sy mninnes 


article is attached) 





Research Procedure 


After dividing the students into three groups, all three groups first accepted the 
pre-test of the Understanding test. After that, the Animation group first accepted the 
animation viewing, the Educational Video group first watched the educational video, 
and the Science Comics group first read The King of Science Experiments (10). After the 
first step was finished, all students would conduct the post-test of the Understanding test. 
After the post-test, each group needed to watch or read the other two media, for example: 
The Animation group needed to watch the educational video and read science comics. 
After watching or reading all the media, the AEJO questionnaire would be implemented. 
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Figure 1 
The Summary of Research Procedure 


Animation Educational Science comic 





group video group group 


' 
v 


Pre-test of Understanding test 


v 


























Watch the 
educational 
video 


Watch the 


animation 


Post-test of Understanding test 


1 
i 
v 


Read science 


comics 





















Watch educational Watch animations Watch animations 





videos and read and read science and educational 





science comics comics videos 


! 
Ub ‘ 
AEIO questionnaire 


Data analysis 


Assessment instruments 


(1) The AEIO questionnaire was adapted from a Scale for Science Edu- 
Communication (Wu et al., 2019). The scale had undergone statistical 
tests and had proven to be a valid and reliable instrument to measure the 
effectiveness of science communication. The validity of AEIO questionnaire 
was established by expert judgment, and we used SPSS 17 to run test- 
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retest analysis of AEI dimensions. The Pearson's correlation coefficient was 
0.916 > 0.7, so the AEI questionnaire had a high degree of reliability. The 
data of Opinion formation was qualitative, so the reliability test was not 
performed. 

(2) This study designed 31 items of heat transfer at first, and then 49 students 
in the third grade were chosen to undertake the preliminary examination. 
According to the results of preliminary examination, some items were 
rejected due to their poor discrimination index. Finally, there were 16 items 
in the Understanding test. 


Table 2 
Question Types of AEIOU and Number of Items 

















AEIOU Dimensions Number of Items Question Type 
Awareness 6 

Enjoyment 6 5-point Likert scale 
Interest 6 

Opinion formation 1 Open-Ended 





Yes/No Questions and Multi- 


Enceleanany i ple-Choice Questions 





Research Results 
AET Questionnaire 


A 5-point Likert scale was used to measure AEI dimensions. The scoring of AEI 
was from | point (completely disagree) to 5 point (completely agree) with a higher total 
score designating higher levels of AEI. Based on the above calculation of the average 
score and the analysis result, it was found that the average score of the AEI dimensions 
was above 4. This data showed that no matter what kind of media teaching, students had 
a positive attitude towards AEI dimensions. Among them, Awareness had the highest 
score of 4.241. This study also analysed the impact of three media on AEI dimensions. 
Among the three media, the animation group got the highest average score in the two 
dimensions of Awareness and Enjoyment, and the science comic group had the highest 
average score in the Interest dimension. 
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Figure 2 
An Average Score of Different Media on the AEI Questionnaire 
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Open-Ended Questions of Opinion Formation 


The question of Opinion formation was that” if you have a house, what can you 
do to make the people feel cooler in the house? Please try to explain from the installation 
of air-conditioning, the treatment of the wall, the position of the window, etc.” After 
integrating the answers, it was found that because the third-grade students had a limited 
vocabulary and insufficient discussion skills, the proportion of answers was not high and 
only short words were answered. The answers of Opinion formation were coded and 
analysed, they were divided into heat conduction group, heat convection group, heat 
radiation group and error groups. 


Table 3 
Analysis of the Answers of Opinion Formation 








Number of : 
Group Examples of answers Studenis Ratio 
Heat Conduction N/A 0 0% 





The air-conditioning is installed on the ceiling, above the 
Heat Convection room, and windows are installed on the left and right sides of 49 29.7% 
the house, etc. 





Heat Radiation — Painted white walls 33 20% 





Did not write, air-conditioning installed next to the TV, win- 


erat dows installed next to the door, etc. 


83 50.3% 





Note: The question can have more than one answer, so the calculation of the ratio is the number of 
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From the results, we know that students were more familiar with the concept of 
heat convection and have a higher rate of correct answers. However, the students may 
be influenced by the question, so they did not answer in the aspect of heat conduction 
at all. Basically, the third-grade students may be limited by insufficient life experience, 
insufficient language skills, insufficient scientific knowledge, and other factors, so their 
performance in Opinion formation was bad. 


Understanding Test 


In order to compare the post-test scores influenced by the three groups of media, 
we used the analysis of One-Way ANOVA (Lin, 2014) to determine whether there were 
any statistically significant differences between the means of three groups. After the 
analysis of One-Way ANOVA (by SPSS 17), the F value was 0.208, and the significance 
value was 0.812(p = .812) which was above 0.05. Therefore, there was no statistically 
significant difference in the three groups of media. Thus, the three groups of media had 
no significant influence on the post-test scores of the Understanding test. 

Then, we used the analysis of the Paired-Sample t Test (Lin, 2014) to compare the 
influence on post-test scores by each medium. In the animation group, the significance 
value was 0.008(p = .008) which was below 0.05, and the average post-test score 
was higher than the average pre-test score, indicating that watching animations can 
significantly improve the understanding of science. In the educational video group, the 
significance value was 0.03(p = .03) which was below 0.05, and the average post-test 
score was higher than the average pre-test score, indicating that watching educational 
videos can significantly improve the understanding of science. And in the science comic 
group, the significance value was 0.047(p = .047) which was below 0.05, and the average 
post-test score was higher than the average pre-test score, indicating that reading science 
comics can significantly improve the understanding of science. 

Integrating the above data shows that no matter what kind of media can improve 
the participants' understanding of science. 


Brief Summary 


To summarize the above results, we can know that the animation group get the 
highest average score in the two dimensions of Awareness and Enjoyment, and the 
science comic group have the highest average score in the Interest dimension. On the 
dimension of Understanding, the analysis results of the three media show no significant 
differences, meaning that there is no difference in the impact of the three media on the 
post-understanding test. Afterward, the Paired-Sample t-Test of each media was carried 
out. The results show that the P-value of each material is less than 0.05, and all of the 
average scores of the post-test are higher than the average score of the pre-test, meaning 
that each media has a significant improvement in student's Understanding. As for Opinion 
Formation, because the subjects are young, they may have insufficient life experience, 
insufficient scientific knowledge, poor language skills, etc., which results in very few 
answers, or more than half of the students did not answer. In conclusion, the third-grade 
students do not perform well in Opinion Formation. 
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Conclusions and Implications 


The spread of COVID-19 has caused a high demand for online learning or self- 
learning at home. We focus on evaluating which media is most suitable for self-learning 
for the third-grade students. To effectively evaluate the learning effectiveness, this study 
uses the five aspects of response to science communication as the evaluation framework. 
These five responses are: Awareness, Enjoyment, Interest, Opinion formation, and 
Understanding. 

In this research, three types of media have been discussed: animation, educational 
video, and science comic. The first two are network resources, while the last is physical 
books. Throughout the experiment, the teacher played the role of observer, and was 
no longer the transmitter of knowledge. Only in this way could it be in line with the 
self-learning situation. Finally, the results pointed out that after the students used the 
three media, the third-grade students had positive feedbacks on the three dimensions of 
Awareness, Enjoyment, and Interest, among which Awareness scored the highest. Next, 
this study analysed the influence of the three media on AEI. Among them, the animation 
got the highest score in the two dimensions of Awareness and Enjoyment, and the science 
comic group got the highest score in the dimension of Interest. As for the dimension 
of Opinion formation, because the third-grade students are relatively young and have 
insufficient life experience and scientific knowledge, the response is poor. Nonetheless, 
all three media can improve students' understanding of science, and there is no difference 
between them. 

In summary, the three types of media showed the effectiveness of self-learning for 
the third graders. Especially the animations proved to be the most suitable for the third- 
grade children to conduct self-learning. For the older students, the media that is suitable 
for self-learning may be changed, which requires further research. 
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Abstract 


In the era of universal and compulsory education, in which attention is paid to the ability to think 
scientifically, there should be no room for unscientific views. However, unscientific theories often 
appear in the media, and they find numerous supporters. Therefore, it was decided to investigate 
which of the common beliefs about the "chemical vs. natural" pair are believed by Poles. And 
whether belief in unscientific myths depends on gender, age, level of education or its type. Check- 
ing these relationships will allow, inter alia, to evaluate the effectiveness of science education 
in Poland. Within 4 years, the beliefs of 2,473 people were examined. The obtained results show 
that the universality of education does not prevent misconceptions. There was also no correlation 
between the correctness of the answers to the questions on age, gender, education level or its type. 
It seems that the way science is taught should be completely modified in such a way that students 
can distinguish truth from myth. 

Keywords: science misconceptions, common beliefs, fake news 


Introduction 


Today, in many countries, compulsory education is considered a right for every 
citizen. It aims to level out the educational differences between citizens and to provide 
a minimum level of knowledge for all citizens (Universal Declaration of Human Rights 
art. 26, Convention on the Rights of the Child art. 28). The main goal of the Europe 2020 
strategy was also to increase the level of education of Europeans (increasing the propor- 
tion of people aged 30-34 with tertiary education in Europe to at least 40%). Recom- 
mendation of the European Parliament and the Council of the European Union on Key 
Competences for Lifelong Learning (2006, 2018) describes in detail what competences 
Europeans should possess upon leaving school. 

One of them is the so-called scientific competence. They refer to the ability and 
willingness to use existing knowledge and methodology to explain the natural world, 
formulate questions and draw evidence-based conclusions. In the case of the natural and 
exact sciences, the necessary knowledge covers the main principles governing the natu- 
ral world, basic scientific concepts, theories, principles and methods, as well as an under- 
standing of the impact of science and human activity on the natural world. These compe- 
tences should enable individuals to better understand the benefits, limitations and risks 
of scientific theories and applications in societies in general (linked to decision making, 
values, moral issues, culture, etc.). Skills include understanding science as a process of 
studying nature through controlled experimentation, the ability to use technological tools 
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and devices and scientific data to achieve a goal or make a decision, or draw a conclusion 
based on evidence, and a willingness to give up one's own beliefs if they contradict new 
scientific discoveries. Individuals should also be able to recognize the necessary features 
of scientific conduct and be able to express the conclusions and reasoning that led to 
them. Competences in this area include attitudes of critical understanding and curiosity, 
respect for ethical issues and promoting both environmental safety and sustainability, in 
particular with regard to scientific and technological progress in the context of the indi- 
vidual, his family and community, and global issues. 

It, therefore, seems that educated people should not believe in unscientific theo- 
ries. However, despite the fact that in Poland the percentage of people with higher edu- 
cation is 45.7% (for the population aged 30-34) and 21% (for the group of people aged 
25-64), many Poles believe in non-scientific, and many of them allow ads to deceive 
themselves. 

International studies on the correlation between scientific knowledge and non-sci- 
entific beliefs most often concern the relationship between science and religion. So, i.a. 
belief in creationism or Darwinism (Allmon 2011; Bishop, 2007; Branch, 2008; Cor- 
nish-Bowden & Cardenas, 2007; Brown, 2010; Plutzer & Berkman, 2008; Williams, 
2009) or belief in the origin of the universe (De Carvalho, 2013; Fisher, 2006; Gleis- 
er, 2005). Other non-scientific approaches are studied less frequently - one of them is 
chemical vs natural opposition (Rozin, et al. 2004; Li & Chapman, 2012; Chouakea & 
Friedman, 2012). And this problem is very common in everyday life. In colloquial con- 
versations and publicity or ad, we meet the opposition: chemical or natural. However, 
is this opposition scientifically justified? From a scientific point of view, there is no 
difference between “natural” and “synthetic” versions of a chemical. Very often people 
think that “synthetic” chemicals (it means - chemicals made in a lab) are not as good for 
them as their “natural” equivalent. This is a complete misunderstanding of the basics of 
chemistry. Chemical molecules have the same properties, whether they were created in a 
laboratory or in a living organism. It seems strange that after 5 years or more of studying 
chemistry in school, misconceptions of this type can appear in people's minds. There- 
fore, it was decided to study the beliefs of ordinary people. It was decided to investigate 
whether belief in unscientific theories is common among Poles. And whether it depends 
on the level and type of education, age and gender of the respondents. 

The main aim of the study was to check the effectiveness of science education in 
Poland. It was hypothesized that younger, better-educated people would not believe in 
unscientific myths. It was also assumed that people with natural science education would 
choose the correct answers more often than other people. 


Research Methodology 


To investigate the common beliefs of people about ‘natural and chemical': drugs, 
cosmetics, food preservatives, etc., the research was conducted for a period of 4 years 
from March 2016 to February 2020. 

The attitude of the respondents to 19 common beliefs - myths that often appear in 
the media was examined. People with secondary education should have no problem sep- 
arating truth from falsehood. (In Poland, education is compulsory up to the age of 18.) 

This article describes only the results of 7 myths about the conceptual opposition: 
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chemical - natural. These are the myths: 
e All natural substances are healthy, 
Herbs are safer than medicines, 
Brown (cane) sugar is healthier than white sugar, 
Vitamin C helps with colds, 
The homeopathic remedies work, 
All chemicals are harmful, 
You can eat vitamins unlimitedly. 

These questions were separated from each other by questions of a different type. 
The research was carried out in Krakow at the Pedagogical University. The study was 
attended by participants of open lectures (undergraduate, graduate and doctoral students, 
participants of 2nd and 3rd age universities, children participating in educational proj- 
ects at our university), and people associated with them (e.g., family). A study of 2,473 
people was conducted. 

A questionnaire was used as a research tool. The respondents' task was to find out 
how much they agree with a given myth. The Likert scale was used as the measurement 
strategy. Because thanks to it you can get knowledge about the degree of acceptance of 
given views by the respondents. The survey had five answers: I fully agree, I agree, I 
have no opinion, I don't agree, I completely disagree. 

The surveyed sample reflected the percentage of society in Poland. 62.8% of the 
respondents were women (which is consistent with the statistical data in Poland, wom- 
en constitute 58% of students, and at universities of the 2"4 and 3" age they constitute 
as much as 86%). Most of the respondents were undergraduate (53.0%) and graduate 
students (28.5%). 12.1% were PhD students. 38.3% of respondents had a humanistic 
education, 37.3% strict education or technical and 13.5% natural science education. 


Research Results 


Table 1 summarizes the results of answering the questions. The analysis of the 
obtained data showed that in three (out of seven questions) the respondents answered 
incorrectly. And in one question they have no opinion. However, even in the questions 
where the majority of the respondents give the correct answer, the percentage of wrong 
and "undecided" answers is high. 
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Table 1 
Percentage of Answers to Selected Questions (Concerning the chemical / natural 
opposition). 
Responses of the study participants 
; | full | | have no | don't | completely 
Investigated myths ip ae’ aioe opinion ont agree disagree 
All natural substances are 11.73 26.20 914 35.30 17.63 
healthy 
Hemera nay 13.30 34.86 20.30 25.03 651 
medicines 
i el all 22.12 44.36 19.17 9.70 4.65 
healthier than white sugar 
Vitamin C helps with colds 32.15 49.09 9.83 7.04 1.90 
The homeopathic remedies 5.05 45.00 56.41 12.58 10.96 
work 
All chemicals are harmful 4.89 14.56 19.69 41.57 19.29 
Racal SaL Vem 4.08 9.38 12.05 39.06 35.42 


unlimitedly 


Note: Responses considered correct are shown in bold. The most common answer appears on a grey 





background. 


The most correct answers were given to the question: “You can eat vitamins un- 
limitedly”. However, even in this case, the percentage of people who disagreed with 
this opinion is around 25%. The second question with the highest number of correct 
answers was: “All chemicals are harmful”. However, in this case, as many as 39% of 
respondents did not indicate the correct answer. The attitude of people who "agree" to 
people who "strongly agree" is also different (for the vitamin C question it is 1.1 and for 
the question of the chemicals 2.2). The third question with a greater percentage of correct 
answers than the others is: “All natural substances are healthy”. However, in this case, 
the percentage of people not choosing the correct answer is very high (about 47%). In 
this case, the ratio of people who "disagree" and "strongly disagree" is 2.0. Among the 
three questions that most of the respondents answered incorrectly, the question "Vitamin 
C helps with colds" had the most incorrect answers. 
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Figure 1 
Percentage of Grouped Answers to Particular Questions 





@ agree | have no opinion don't agree 
100,00 
81; ee 
75,00 
48,16 
50,00 
37,93 
T 
25,00 j 20) 05 19,4 
: I > 13,46 
0,00 i ip 
Allnatural Herbs are safer Brown (cane) va c Chains The All chemicals You can eat 
substances are than medicines sugar is homeopathic are harmful vitamins 
healthy healthier than remedies work unlimitedly 
white sugar 


Note: (answers "I agree" and "I strongly agree" were grouped with the answers "I agree" and answers "I 
disagree" and "I strongly disagree" were grouped with the answers "I disagree"). 


The Spearman coefficient was calculated for the obtained data. In the part con- 
cerning the respondents, a moderate, positive relationship between the age of the respon- 
dents and their education (7, = .53) and a weak relationship between sex and the type 
of education (r, = .27) were obtained. In none of the seven questions, a correlation was 
found between the type of answer to the question and gender, age, education, and type 
of education. It was found moderate dependencies between the answers to the questions: 

¢ All chemicals are harmful: You can eat vitamins unlimitedly (r=.46); 

* All natural substances are healthy: All chemicals are harmful (7 =.44); 

¢ All natural substances are healthy: Herbs are safer than medicines (7=.41). 
Weak relationships were found between the answers to the questions: 

¢ All natural substances are healthy: You can eat vitamins unlimitedly (7=.34); 

¢ Brown (cane) sugar is healthier than white sugar: Herbs are safer than medi- 

cines (7.=.26); 

¢ Brown (cane) sugar is healthier than white sugar: Vitamin C helps with colds 

(r=.26); 

* All chemicals are harmful: Herbs are safer than medicines (7 =.26). 

No correlation was found between the answers to the remaining questions (Table 2). 
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Table 2 
Spearman's Coefficient for Individual Pairs of Questions 





All Herbs Brown Vitamin The All chem- You 
natural are safer (cane) C helps home- icals are can eat 
sub- than sugar is with opathic harmful vitamins 
stances medi- healthier colds remedies unlimit- 
are cines than work edly 
healthy white 
sugar 









All natural 
substanc- 
es are 

healthy 






AM 18 08 14 44 34 









Herbs are 
safer than 41 
medicines 








Brown 
(cane) 
sugar is 
healthier 
than white 
sugar 


18 








Vitamin C 
helps with 08 
colds 








The ho- 
meopathic 
remedies 
work 


14 





All chem- 
icals are 44 
harmful 





You 

can eat 
vitamins 
unlimitedly 


34 





Discussion 


The results obtained show that, in the opinion of the general public, there is a 
belief that natural substances are superior to synthetic (chemical) substances in terms of 
effectiveness and safety in human health matters, although this is not true. Toplis (2002) 
obtained similar results to those obtained in these studies. It seems that the influence of 
the media promoting products containing the so-called natural substances as pro-health 
products have a great influence here. For example, Valkenburg et al. (2016) write about 
the power of the media on human attitudes and beliefs. 
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However, no influence of the degree of education or its type on the beliefs of the 
research participants was noticed. Similar results, i.e., the lack of influence of the lev- 
el of education on people's attitudes (in this case, regarding climate change), describe 
Funk (2017) in the article How much does science knowledge influence people's views 
on climate change and energy issues? Similar research results were obtained by Impey 
et al. (2012). It seems that the summary of the considerations in the article "Belief, 
Knowledge, and Science Education" (2001, p. 349) still remains valid: "... we must shed 
light on this subject from a variety of sources — theoretical and empirical, philosophical 
and psychological — to advance our understanding of knowledge and beliefs and their 
influence on science learning". 

Perhaps the explanation of the facts of belief in unscientific myths, despite the 
acquired scientific knowledge, can be found in the laws of Jost (1897). Although these 
laws were made a long time ago, it seems they are still valid today. 

The first law states: As time goes on, the power of associations of the elders weak- 
ens more slowly. So, after many years, a person remembers the original associations 
associated with a given concept. So, for example, non-scientific explanations of parents 
or grandparents or information from advertising. That is why it is so important that the 
first explanation that a child meets in the process of environmental education is correct. 

The second law states: If two associations are of equal strength but one is older 
than the other, then the repetition will favour the older association more. 

This law also indicates the importance of the first connotations associated with a 
given concept. As the first associations with natural phenomena remain in memory for 
the longest time, their improper shaping may lead to a negative transfer at later stages of 
education and misconceptions in adulthood. 


Conclusions and Implications 


The obtained results indicate that a large percentage of the society does not have 
basic knowledge about the health properties (or harmfulness) of natural and synthetic 
substances. Despite the fact that in the Polish core curriculum for Nature, Biology and 
Chemistry, we can find many topics related to this issue. For example: In the core curric- 
ulum for elementary school for the subject Nature there is an entry: the student describes 
the effects of substances harmful to health, recognizes poisonous plants and poisonous 
animals. The core curriculum for Biology includes the entire section on "Chemistry of 
life", where attention is drawn to the lack of distinction between substances of natural 
origin and synthetic substances - obtained in a laboratory. And in the section "Diges- 
tive system and nutrition" is discussed, inter alia, digesting sugars and using vitamins. 
In the core curriculum in Chemistry, in the section "Chemical substances of biological 
importance", the properties of the substances that are the main components of everyday 
products are described. And in the section "Water and water solutions", the student per- 
forms calculations using, among others, concepts: solubility, percentage concentration. 
Therefore, it seems that after completing primary school education, the student should 
acquire knowledge that would allow him to correctly answer the questions asked in the 
questionnaire. However, as the results show, this is not the case. 

The obtained results are a big challenge for people responsible for education, 
including teachers. The results show that non-scientific ideas are widespread among peo- 
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ple and resistant to standard teaching methods. The question then arises: How and what 
to teach students to become informed citizens who vote / choose wisely on science and 
technology? 

There seems to be some action to be taken to combat unscientific myths and to 
ensure that scientific thinking is at the heart of our teaching. Policy makers and curric- 
ulum developers must ensure science education as early as possible in kindergartens or 
primary education. False, unscientific perceptions instilled by the media or parents and 
naturally occurring in younger children will be very difficult, if not impossible, to correct 
at a later date. Curriculum makers and policymakers should also introduce a discussion 
and explanation of the most damaging anti-science myths into curricula. 

Children in kindergartens and schools should be taught the 'scientific reasoning’, 
for example by making extensive use of the IBSE or learning to discuss and argue. 
Teachers should be given the tools to combat unscientific theories, as well as a way to 
deal with those who hold such views. Scientists should be more involved in educating 
the public. They should avoid inappropriate and imprecise language; they should react 
when unscientific myths appear in the media. Working together can help prevent science 
misconceptions. 
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Abstract 


The National Defense University (NDU) trains officers to develop their academic and professional 
skills. To accomplish this, the university offers two mandatory courses on methodological 
training for military technology students for master level education. The first course was 
theoretically oriented, and the second course was practically oriented. These both master-level 
methodology courses emphasize practice oriented mathematical skills, which officers use in 
their operative decision-making and statistical analysis. This study focuses on student-centered 
learning methodologies linked to teachers’ observations from current and previous course 
implementations. Results in this study described the outcome from the first run of the revised 
curriculum. We collected data from students’ course reports and the university's standard student 
evaluation of teaching (SET). According to the SET; the course 2 which was practically oriented 
course, where groups worked on more significant projects gained higher value among students. In 
conclusion, we recommend that teachers continue using student-centered learning methodologies 
to technical students as much as possible. Theoretically underscored courses should also contain 
more practical examples. 

Keywords: distance education, flipped learning, learning by doing, research methodology, 
student-centered learning 


Introduction 


Theoretical knowledgebase is an essential aspect of technical teaching. However, 
in mathematics courses theoretically oriented courses require intense content design to 
be inspiring for students. Is it just about enhancing students’ thinking skills, particularly 
abstract thinking? As late as the early 20th century, for about 50 years, Neovius- 
Nevanlinna mathematics textbooks were used almost exclusively in high schools. These 
books were considered consistent but demanding (e.g., Lehto, 2004). At that time, 
learning outcomes were typically such that one-third of students did not even reach 
the accepted grade. According to Nevanlinna (2021) the learning goals implemented in 
these textbooks were possibly too abstract for school children at that time. Nowadays 
mathematical knowledge is an essential component of experimental research. It is not 
merely a tool, but a vital part of exploratory questioning and a general methodological 
approach used in both quantitative and qualitative research (Aczel, 2006). Earlier in 
NDU, in officers’ methodology education, master-level students appreciated a short, 
one-day-long intervention experiment around Word Math Day (Rissanen & Mutanen, 
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2019). In general, master-level technical students in NDU are open to any non-traditional 
math education. 

In 2020, NDU’s curriculum for master students changed a little bit, and some 
courses merged. The standard general line only has subject specification for the master 
thesis and formal support for discipline-specific knowledge linked to the thesis. 
According to the update in the curriculum, the Department of Military Technology 
gives two research methodology courses that support each other. This article presents 
those two updated formally Bologna process-based military technology courses where 
mathematical knowledge and practical skills are major learning items. The first course 
gives basic knowledge about traditional statistical analysis tools and includes a short 
session on how to make research in general. The second course teaches tools that would 
help to evaluate research settings. The preliminary modeling usually works as an advising 
tool during technical research. Therefore, the first course is more formal and theoretically 
oriented than the second course. This study aims to understand how students responded 
towards each of the two research methodology courses, namely Methodology course 1 
(theoretically emphasized distance learning and class learning course) and Methodology 
course 2 (practically emphasized distance learning alone). While the first course is more 
theoretically oriented, the second one is more oriented towards the technical aspects of 
practical research methodology. As the COVID-19 pandemic changed routines, the first 
course used more lecturing, while the second course employed more distance education 
principles. 


Research Aims and Questions 


The purpose of this study was to investigate the learning efficiency and also 
student satisfaction in general between these two courses. 
RQ 1: Students ’attitudes towards mathematics as a research method and how do students’ 
responses vary between these two courses? 
RQ 2: How gained results of student responses should be noticed in future course design? 


Theoretical Frames 
Student-centered Learning 


In student-centered settings, the responsibility for learning shifts naturally falls 
to the student (Wright, 2011). Students cooperate and take intense responsibility for 
their studies. Master-level students majoring in scientific subjects on NDU’s technology 
program have different (individual) instructional needs. Therefore, the current study 
explores how relatively practice-oriented courses affect students’ motivation and 
attitudes towards physics and technology as professional tools. On the other hand, the 
more traditional starting course would be a good reference for the study. Both courses 
include support for individual learning. 

The literature suggests that student-centered learning can be approached e.g., using project- 
based learning (PBL). In the 21st century, this concept is used to revise curriculums. With 
a student-centered learning approach, students can experience real-life situations inside 
the classroom (or equally in their distance learning premises). Instructors are supporting 
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students on their journey to achieve learning. At all academic levels, small groups or 
teams working on projects are a mandatory arrangement. However, this arrangement 
alone does not allow higher levels of critical thinking. A traditional “one-size-fits-all” 
approach will not work for all types of education (Zmuda, 2009). 


Action Research 


Action research encourages researchers to participate in everyday life of the 
focus organization. Attendance combines with and influences subject analysis (Kemmis 
& McTaggart, 2013). Processes are not as straightforward as the sequential parts of 
independent design, operation, observation, and reflection. The described modules 
may overlap, and the original plans may become irrelevant based on experience and 
new information. Activity-related research can be reflected as an experiential learning 
approach, with sophisticated methods and enhanced knowledge interpretations based on 
the understanding of previous cycles (O’ Leary, 2004; see Figure 1). 


Figure 1 
Action Research Cycle. Ideas of Continuous Research and Enhancement 
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O'Leary 2004 concept for cycles in action research 


Flipped Learning and Distance Education 


Acase study is a way of conducting research. It can be called a qualitative research 
approach, but a case study is not synonymous with qualitative research. The case study 
approach is not in itself a standalone research method. It isn’t easy to give a general or 
comprehensive definition of a case study because there are different types of case studies 
(Crowe et al., 2011). In this work, we use it as a research strategy. 

The flipped classroom refers to teaching with active and blended learning styles. 
The learning material is provided before any handling in lectures. This technique leads 
to integrating face-to-face and online teaching techniques enabling students to engage in 
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meaningful and communicative learning and critical problem-solving. Therefore, course 
instructors may act as facilitators, guiding students in the discussion individually or in 
groups during class time. The implementation of flipped learning in a higher education 
course improved students’ achievements and attitudes towards learning (e.g., Garrison 
& Akyol, 2009). 

Ganesh (2021) presents how flexible a Moodle platform can be for this type of 
flipped learning. In his experimental videos and other study materials, he distributed 
texts through MOODLE platform. Learning targeted student-centered learning was 
organized underlining working in pairs or small teams. 

Distance education has a long tradition, especially in Australia and the US Navy 
(e.g., Arenas, 2005). Due to the sudden and life-threatening COVID-19 pandemic, a 
large part of traditional university education has stopped. In Italy, Giovannella (2021) 
studied how the rapid transformation to distance education as the only choice, affected 
students’ opinions. Although students seem to miss physical campus activities, the switch 
from physical to virtual environment was taken in a positive way. Moreover, a large part 
of the present generation of university students seems to be ready for novel educational 
activities. 


Research Methodology 
Data Collection 


We used standard local SET questionnaires in PVMoodle for data collection, data 
comparisons, and data storage. Observations focused on common course impressions. In 
addition, we analyzed personal estimates of learning results, motivational aspects, and 
free text impressions. The questionnaires involved a 5-point Likert scale. When we com- 
bined the quantitative data from the questionnaires with the qualitative data—such as 
data gathered using open-ended questions, participant observations, and interviews—the 
questionnaires’ validity improved, and our results became more accurate. For Figure 2, 
we selected only eight most analyzable questions from the standard questionnaire. 

The answers gathered are to the following questions or notices: 

1) achieved the goals set for this course. 

2) I was active during the course? 

3) Was the overall ambiance during the course supportive of your studies? 

4) How well did the instructors master the subject matter of the course? 

5) Give an overall grade for the instructors. 

6) Did you learn new information/skills? 

7) Did the evaluation of this course support your learning? 

8) Give an overall grade for this course. 

The number of students who voluntarily participated in the evaluation was 21 for 
the course | and 12 for the course 2. 
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Research Material 


Course implementation 
Teaching in the first course (Hybrid course with class learning and distance learning) 


The first course, which included research practice and statistical methods, was 
organized into two parts. According to the course plan (a detailed plan expressed in the 
weekly program), mathematical content and statistical methods were taught intensively 
in two-week period. Teaching methods included lecture teaching, illustrative demonstra- 
tion examples, and voluntary and supervised practice. Instead of fully present or distant 
education, a hybrid version was utilized. Therefore, remote participation was possible 
but not too comprehensive. The assessment consisted of five sets of tests based on scored 
exercises (1.e., 23 calculation tasks), of which 60% of the maximum points had to be 
obtained for the approved performance. Along with the course progress, 21 out of 22 
students passed this formative test. The one who failed had an accident which prevented 
further participation along the spring term. All students responded to the feedback ques- 
tionnaire after the professor’s reminder. This course aims to acquire the skills related 
to Pro Graduate research and consists of exercises and knowledge creation in group 
works. Observations from student evaluation of teaching (SET) data complemented this 
knowledge. 


Teaching in the second course (distance learning) 


The second course, which consisted of practical modeling methods and simulation 
tools, took place during spring term in three weeks period. The instructional structure 
of course 2 was similar to the previous years’ simulation and modeling course, which 
had been carried out eight times with good feedbacks from students and with only minor 
modifications. It consisted of three overlapping teaching methods, namely distance lec- 
tures, supervised exercises, and unsupervised exercises. At the end of the course, student 
groups presented their unsupervised exercises. The final report consisted of documen- 
tation and the functional version of the group’s own specific modeling realization. The 
course used simulation and mathematical modeling as a research method. 

The first week (in the 3 weeklong course module) content was about random 
numbers and distributions, IF-THEN structure and reasoning, Lanchester equations, 
Markov chain, Monte Carlo simulation, and Queuing theory. Two distance lectures on 
machine learning, artificial intelligence in the Defense Forces, and general machine 
learning methods and the skills required for their use were also presented. Learning 
exercises called seminar work were also published. Most of the seminar work topics 
came from the students themselves in connection with their master’s theses. 

Course content in the second week was about mathematical models to support 
decision-making, value tree analysis, analytical hierarchy process (AHP), and 
optimization methods. 

In the third course week, students focused on completing shared homework as 
well as seminar work. Students were given personal guidance using remote connections 
and according to their research needs. By the end of the course, the exercises were 
complete. In the last teaching event, the students presented their seminar work in groups. 
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The course aims to help students understand the methods taught and apply them. Based 
on the questions actively asked by the students, the work done in the final seminar, and 
the student feedback, we can conclude that the goals set at the beginning of the course 
have been met. 

Learning took place remotely using Zoom and PVMoodle. Also, additional 
communication was done via Skype and email. Using Zoom for cloud-based video 
conferencing is not new (e.g., McCoy, 2015), but due to its flexibility to enhance distance 
education during the COVID-19 pandemic, both courses in this study utilized where 
applicable. 


Research Results 
Analysis Based on Student Feedback 


Students are familiar with the National Defense University’s standards for student 
evaluation of teaching (SET). However, the validity of student evaluation of teaching 
(SET) in universities depends on which purpose it has been utilized. An early and limited 
study made in the UK context (Shevlin et al, 2000) demonstrates that teaching ability 
can influence evaluations. Moreover, they name other factors like student profile and the 
physical environment. To overcome some of the side effects, they included new elements 
related to the lecturer’s charisma. With a student-centric approach, opinions of teachers’ 
charisma become less critical. On the other hand, e.g., Athtyaman (1997) claims that 
student satisfaction and service quality could be a way to construct a value from the SET 
data type of information. 

Feedback data from students with SET provides the necessary information for 
instructors on how to streamline teaching protocols, but it gives only a few tools for 
making significant educational improvements. Therefore, development work in this field 
requires more than just empirical evidence of learning results and students’ wishes and 
opinions. Technology development also offers this era new opportunities to organize 
education in a rewarding way, and genuine innovations in this area require more than 
gathered formal feedback. The responsibility to re-engineer education creatively creates 
a continuous challenge for instructors and organizations. For that purpose, student re- 
ports and written statements in SET are indirectly utilized. 
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Figure 2 
Students’ Evaluation of the NDU’s Research Methodology Courses 2021 (Course 1, 
N:21; Course 2, N:12) 
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Discussion 


Student feedback from Course 1 was generally little over average in factors 
describing student activity. The professional skills of teachers were good. On the other 
hand, based on the feedback survey, it is worth adding mathematical details to the 
teaching material in the learning material portal. This is a clear development plan for 
statistics in the subsequent implementation. The feedback from the Courses 1 and 2 
was at an excellent level. However, scheduling is still worth refining for the courses. 
Both courses included exercises. The statistical methodology in both courses was 
demanding but also relatively straightforward to apply. On the other hand, according 
to students, modeling had more connections to daily life. But also, the teachers needed 
more detailed mathematical formulation of the set problem or intermediate processes 
towards the solution. According to the data presented in Figure 2, the overall attitude 
toward learning was higher in the later course (2, practically oriented student-centered 
course). Due to nine lacking answers in course 2, the numeric data can be utilized for 
profiling with other data. According to the SET data with open question answers, course 
2 was perceived more student oriented. In this course, students could suggest their final 
works and worked collaboratively under their teacher’s supervision. Because of this, 
students appreciated it more. 


Conclusions 


A balance between rigor and practice is essential. Students at NDU have always 
wished for more practice-based examples. Therefore, it is understandable that they 
prefer a course that emphasizes practice. Distance learning does not mean less student 
satisfaction. Results showed that students were generally more interested in practice- 
oriented student-centric education. This does not mean that theoretical education is less 
valuable. On the contrary, the results showed that educators could inspire students to value 
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theoretical knowledge if they demonstrate the practical applications of such knowledge. 
Theoretically emphasized course (course 1) should also include more real-life related 
examples. However, more research is required to deliver theoretical knowledge in an 
inspirational and student-friendly manner effectively. 
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Abstract 


Examining students’ inclinations to use algorithms and rules to solve a task was a fruitful area of 
research in chemical education in the last four decades. This research aimed to examine whether 
students read the task request carefully, considering its meaningfulness, or they approach it 
mechanically, applying a set of algorithms by default. The research sample consisted of students 
majoring in chemistry teaching at the University of Novi Sad, Faculty of Sciences who were 
in their final year of bachelor studies. The study was conducted during two academic years. 
The main instrument consisted of five quantitative problems, and each of the problems contained 
deceptive information that made the calculation nonsensical. The results revealed that most 
students applied an algorithmic approach without paying attention to the meaningfulness of 
the task requirements. Additionally, it has been shown that students rely heavily on memorizing 
formulas without a proper understanding of underlying concepts. 

Keywords: algorithms, conceptual understanding, quantitative problems 


Introduction 


One of the most important outcomes of chemical education is the development 
of conceptual understanding (Rodi¢, 2018). There are many definitions of conceptual 
understanding suggested so far which are mostly intuitive. This study relies on a definition 
proposed by Holme et al. (2015) which resulted from extensive research and meets the 
most comprehensive requirements of the chemistry education community. It describes 
the conceptual understanding across five dimensions: (i) transfer (student’s ability to 
apply main ideas in chemistry to novel chemical situations), (11) depth (reasoning beyond 
memorization and algorithms), (iii) prediction (ability to explain behaviour of chemical 
systems), (iv) problem solving (reasoning involved in solving problems) and (v) 
translation (ability to translate between scales and representations). However, although 
conceptual understanding is a desirable learning outcome, in practice, students are far 
more in favour of the algorithmic approach. There is strong evidence that students are 
inclined to use algorithmic problem-solving strategies without engaging in conceptual 
reasoning (Cracolice et al., 2008; Lazenby & Becker, 2019). 
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This may be related to the fact that school chemistry around the world is organized 
in such a way that it is subordinated to the success of students in the final examinations 
(Stamovlasis et al., 2005). Therefore, teachers are often forced to dedicate the vast 
majority of their teaching time to practising very complex calculations. In that way, 
students become experts in calculations, and at the same time, they do not understand the 
conceptual basis of the content itself. 

This research aimed to examine whether students — future chemistry teachers from 
Novi Sad, have a mechanical/algorithmic or conceptual approach to solving quantitative 
problems in chemistry. The main idea was to find out whether students read the request 
carefully, considering its meaningfulness, or they approach it mechanically, applying a 
set of algorithms by default. Accordingly, the following research questions were set up: 
RQ 1: Do students possess declarative knowledge regarding some selected concepts? 
RQ 2: Do they activate that knowledge during the problem-solving process, or such 
knowledge is dysfunctional? 


Research Methodology 
Study Sample 


Two generations of future chemistry teachers took part in this research. Both 
groups were in their fourth (final) year of bachelor studies. There were 22 students, 12 
in the academic 2016-2017 and 10 in the academic 2017-2018. Of the total sample, 3 
students were male and 19 were female. Although the total sample was 22 students, it is 
representative since it encompasses all students of the target group (students majoring in 
chemistry teaching at the University of Novi Sad) over two academic years. In addition, 
this study has mainly a qualitative character and the sample cannot have much impact 
on research outcomes. 

All respondents agreed to volunteer in the research. They were explained that their 
answers and results are going to be used for research purposes without any compensation 
and that they can opt-out of the study at any point. 


Research Instrument 


Two instruments were used in the research. The first instrument (T1) was a 
paper and pencil test with four multiple-choice tasks, which were designed to provide 
the answer to RQI. This test examined declarative knowledge such as the knowledge 
of insoluble compounds and metals in water, the relative position of metals in activity 
series, and molar volume. T1 questions are given in Box 1. 


Box 1. Tl questions 
Q1. Which answer includes all the following that are insoluble in water? Circle the letter of the 
correct answer. 


la) AgCl, NaNO,, Ba(NO,), b) AgCl, BaSO,, Al(OH), 

c) Ba(NO,),, AgCl, KCl d) KCI, AgCl, Al(OH), 

Q2. Circle the letter of the correct answer. Which of the following metals does not dissolve 
| either in hot or cold water, under standard pressure? 
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Q3. Circle the letter of the correct answer. Which of the gases bellow is product of the reaction 
between copper and concentrated sulfuric acid? 


a)H, b)HS c)SO, d)H,O 
Q4. Circle the letter of the correct statement. 





The second instrument (T2) was a test consisting of five conceptual tasks (CT). 
It was designed to provide an answer to RQ2. Each task contained a piece of deceptive 
information which made the calculation senseless. Namely, students were requested to 
calculate the molar concentration of the solution, but the compound given was insoluble 
in water; to calculate the volume of a product, but the reaction was impossible; to 
calculate the molar volume of a solid substance; to calculate pH value of the solution 
formed by dissolving the metal in water, but the given metal was insoluble in water. T2 
tasks are shown in Box 2. 


Box 2. T2 questions 

(CT 1. Calculate the molar concentration of the solution obtained by adding 15 g of silver 
chloride to 50 cm? of distilled water. (STP) 

(CT2. Calculate the volume of hydrogen formed in the reaction of 6.35 g of copper with the 
corresponding amount of sulfuric acid. (A (Cu)=63.5; A (H)=1; 4 (S)=32; A (O)=16) (STP) 

ICT3. What is the pH of a solution formed by the addition of 0.1 mol of aluminum to a 10 dm? of 


ater? (A (Al)=27) (STP) 
(CT4. How many grams of barium sulphate should be added to 150 g of distilled water to prepare 
la 10% solution (A (Ba)=137) (STP) 
ICTS. What is the volume of carbon needed to react with the appropriate amount of oxygen to 
produce 2.24 dm’ of carbon dioxide? (A (C)=12; A (O)=16) (STP) 





Testing was conducted in 4 phases: (1) February 2017 (T1, generation 2016-2017), 
(11) March 2017 (T2, generation 2016-2017), (iii) February 2018 (T1, generation 2017- 
2018) and (iv) March 2018 (T2, generation 2017-2018). Time for solving T1 and T2 was 
unlimited, and all the tests were carried out anonymously. Data processing was carried 
out in the program Microsoft Office Excel. 


Research Results 
Results of T1 


Figure 1 shows the Tl results for both generations of students. Based on the 
results, it is evident that students in a very high percentage were successful in solving the 
first, second and fourth questions, which were related to the recognition of compounds 
and metals insoluble in water and the molar volume of gases. The lowest achievement 
was accomplished in the third question, where SO, was expected to be recognized as a 
product in the reaction of copper and concentrated sulfuric acid. High achievements on 
TI questions were expected, as students were in their final year which means they had 
the expected declarative knowledge with certain exceptions (Q3). Such results were a 
precondition for the next stage of research, that is, for conducting T2. 
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Figure 1 
Achievements on T1 
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Results of T2 


The results of T2 will be analysed qualitatively on the selected examples of tasks. 
In CT1, students were asked to calculate the molar concentration of silver chloride in 
the solution formed by its dissolution in water. Although by solving Q1 on T1 students 
showed that they know that silver chloride has low solubility in water, in CT1 they 
ignored this information. All students in this study tried to respond to the request of the 
task by applying a standard formula for calculating the molar concentration of a solution, 
and none of the students noticed the deceptive information. Figure 2 shows an example 
of a student's response. 


Figure 2 
An Example of a Student’s Response on CT1 


Calculate the molar concentration of the solution obtained by 


Zadatak 1 adding 15 g of silver chloride to 50 om” of distilled water. (STP) 





Odredite koliéinsku koncentraciju  srebro-hlorida u rastvoru nastalom 
rastvaranjem 15 ¢ srebro-hlorida u 50 em? vode (STP). (A(Ag)"107,8; A(Cl)=35,5;) 


PACA CLV tn 5, tg nt 





The task is easy, one just needs to know the formula 
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In CT 3, it was necessary to calculate the pH value of the solution, which is 
formed by dissolving aluminum in water. In this task, the students did not pay attention 
to the key information - aluminum that does not dissolve in water. Surprisingly, none 
of the respondents linked the insolubility of aluminum in water with its application in 
everyday life, for example in the production of aluminum packaging used for storing 
foods such as milk, yoghurt and other products with a high percentage of water in their 
composition. On the other hand, the students were very successful in solving Q2 where 
they recognized aluminum as a water-insoluble metal. This means that students have 
declarative (factual) knowledge, but that this knowledge is rather passive. An example 
of a student’s response on CT3 is provided in Figure 3. 


Figure 3 
An Example of a Student's Response on CT3 


Zadatak What is the pH of a solution formed by the addition of 0.1 mol of aluminum to a 10 dm” of water? 





Kolika je pH vrednost rastvora nastalog rastvaranjem 0,1 mol aluminijuma u 10 
dm? vode? (STP) (A,(Al)=27) 
Postupak | komentar: 

pe ft Foe uw 


sad 


a] frre 4 ane 


rye OAH 


The task is easy because the concentration of 
[OH’] ions is practically given and knowing that 
we just have to calculate the concentration of 


[H*] ions and then pH. 





In CTS, it was necessary to calculate the volume of carbon, however, since carbon 
is an element in a solid-state under STP, it was not possible to calculate its volume using 
only the data given. The students overlooked this fact and performed the calculation 
assuming that the volume of | mole of carbon is 22.4 dm} as if the carbon under STP 
is a gas (figure 4). As with the previous questions on T1, the students were successful 
in solving Q4 where they showed that they knew that the value of 22.4 refers only to 
gases. However, this knowledge was not applied in CT5, which once again confirmed 
that when solving quantitative problems, students give preference to algorithms over 
logical reasoning. 
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Figure 4 
An Example of a Student’s Response on CT5 


What is the volume of carbon needed to react with the appropricte 


Zadatak £ amount of oxygen to produce 2.24 dm? of carbon dioxide? (STP) 





Kolika je zapremina ugijenika potrebna da u reakciji sa odgovarajucom koli¢inom 
kisconika nastane 2,24 dm? ugljen-dioksida? (STP) (A,(C)=12; A{O)=16) 


Postupak i komentar: C+ Chez exo f Lye 
4 


¥ “ 
Witil¢ 





Through proportion 


end by applying 
the formula it can 


be solved easily. 


Analysis of Students’ Comments 


As can be seen in Figures 2, 3 and 4, students were expected to give a brief 
comment on each CT related to the solving procedure. Students were instructed to 
provide a comment in a blank field reporting on how difficult or easy it was to resolve a 
task. Additionally, they were asked to reflect on each CT; whether the request was clear 
and logical to them, or they found some information missing, misleading etc. 

Selected comments: 

¢ The task is easy, one just needs to apply the formula. 

¢ The task is easy if you know the formula. 

* The task is not a problem if the formulas and the value of V, are known. 

¢ The task is easy because the concentration of [OH] ions is practically given 

and knowing that we just have to calculate the concentration of [H*] ions and 
then pH. 

¢ Extremely easy because one just needs to substitute known variables into the 

formula. 

¢ The task is easy because by applying the formula it can be solved in one step. 


¢ Tcan't remember the formula. 

¢ Tm not sure if I used an appropriate formula. 

e The task is not difficult, but I can’t recall the formula. 

* I forgot the exact formula. I think it's easy, but I can't remember the formula 
I need to apply. 

¢  Ican't calculate, because I haven't used that formula for a long time. 
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What all comments have in common is the word formula. Most ofthe students were 
very confident that they resolved the tasks correctly because they knew the appropriate 
formula. From their comments it is clear that they apply an algorithmic approach — they 
read the text of the task, extract the data, write the appropriate formula and substitute 
known variables into the formula. At the same time, they do not pay enough attention to 
the meaning of the requirements. 


Conclusions 


Based on the results of Tl, it can be concluded that students had certain 
declarative knowledge. All students who tried to resolve the tasks from T2 were using 
known formulas and algorithms. None of the students noticed the concepts that made the 
calculation nonsensical. Based on the comments, it can be concluded that students do not 
pay enough attention to the underlying concepts. 

These results suggest that more attention should be paid to the conceptual 
underpinnings and less to a drill that is based on solving a large number of calculations 
by mechanical learning of steps and by rote memorization of procedures. Students must 
understand the meaning of calculations and their relation to everyday life. For this 
purpose, it is convenient that tasks contain an everyday life context that will enhance 
student motivation, thus enabling the acquired knowledge to be long-term and more 
functional. 
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Abstract 


Pandemic has changed the way education has taken place in Brazil, which has occurred remotely. 
The classes have been taught in the public network especially through lives or WhatsApp. In this 
paper, the way in which the training activities of the supervised internship and the Pedagogical 
Residency in the teaching of Physics in schools took place will be addressed. As a result, the 
abundant use of new information and communication technologies was verified. Resources and 
methodologies such as digital platforms, simulators, gamification, among others, were used by 
pre-service teachers during supervised internship and / or Pedagogical Residency activities. With 
this, pre-service teachers were able to immerse themselves in the current school reality, better 
understanding how teaching has occurred in this period and to propose activities that could help 
to improve the quality of teaching offered in basic education schools and in the recycling of in- 
service teachers, as well as in the motivation of students, too. 

Keywords: pandemic period, pedagogical residency, physics teaching, supervised internship, 
teacher education 


Introduction 


By the end of 2019, COVID-19 had rapidly spread to China and thus to the rest 
of the world (Albuquerque et al., 2020). With this, Brazil and the world are continually 
facing the social, economic, and emotional impacts of the pandemic. In Brazil, federal 
law no. 13,979 (2020), which provided for measures that could be adopted to deal with 
the public health emergency of international importance resulting from the coronavirus 
responsible for the 2019 outbreak, resulted in several other laws, both at the federal, as 
well as state and municipal levels. 

Brazil is a country of continental dimension, composed of 27 Federative Units 
(FU), popularly known as states. There is a very large disparity between these UFs. Piaui, 
one of the Brazilian FUs and whose capital is Teresina, the city where the research was 
carried out, has a Human Development Index of 0.697 (IPEA, 2019), the third lowest 
among all Brazilian FUs and similar to that of Vietnam, well below Brazil's HDI, which 
is 0.778. In this state, in response to law no. 13,979, it can be highlight decree no. 18,884 
(2020), which regulated the aforementioned law at the state level and determined the 
suspension of classes in the state public system and recommended the closure of other 
schools and Universities in the state, as well as churches, bars, restaurants, stores, etc. 
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Decree no. 18,901 (2020), determined the suspension of religious, commercial, 
aesthetic and sports activities in the state of Piaui. Social isolation was decreed with 
a ban on opening economic activities, closing schools and universities, non-essential 
commerce, and public leisure areas, among others, with the exception of essential 
activities, such as cleaning, health, safety, among others. With this, ordinance no. 343 
(2020) that proposes replacing face-to-face classes with classes in digital media while 
the Covid-19 pandemic situation lasts. Through the ordinance, the Ministry of Education 
decides: 


“Art. 1 Authorize, on an exceptional character, the replacement of on-site subjects, in 
progress, by classes that use information and communication means and technologies, 
within the limits established by the legislation in force, by a higher education institution 
that is part of the federal education system, of which deals with art. 2 of Decree n. 9,235, 
of December 15, 2017 (Ordinance no. 343, 2020, p.01, author's translation)”. 


Most of the municipal governments adhered to remote education. Schools have 
had their classroom activities suspended and many states and cities have adopted the 
continuity of classes through the remote system. Thus, the new reality abruptly imposed 
opened up the need to revise the teacher training, in order to make them able to work in 
a dynamic world that requires professionals capable and adaptable to the new demands. 
In this way, one of the things that would need to be revised was the supervised internship 
(SI), in which the future teacher immerses in schools, being, therefore, a very rich 
moment of learning, which we will deal with in the next section. 


Teacher Education, Supervised Internship and Pedagogical Residency 


Supervised internship is, according to 1* article of law no. 11,788 (2008, author's 
translation), “supervised school educational act, developed in the work environment, 
which aims at preparing students for productive work who are attending regular education 
[...]”. This law adds in its items the internship is part of the pedagogical project of the 
course and aims the learning competences proper to the professional activity and the 
curricular contextualization, aiming at the development of the student for the citizen 
life and for the work. In SI the pre-service teacher carries out observation and teaching 
activities. 

SI is seen by a lot of people as the practical part of the course. Pimenta and 
Lima (2015) have put SI as a theoretical and practical activity. So being considered, 
it is interesting that research is developed to optimize this moment. André (2015) has 
observed that there is a consensus in the educational literature, both in Brazil and abroad, 
that research is an essential element in the professional training of teachers. Severino 
(2008) affirms that one learns to research by researching and, during graduation, this 
research is carried out concretely in the Course Conclusion Monography (CCM) and in 
the scientific initiations. Rodrigues and Arroio (2018) have argued that the SI period can 
also be a fertile period to carry out research. 

Pimenta and Lima (2015) have commented that SI is intended to approach reality, 
and the internship is not a practical activity, but a theoretical one, which implements the 
teaching praxis, which is the activity of transforming reality. The authors (idem) also 
comment research in the internship is a possibility in the training of the intern as a future 
teacher. 
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In addition to the SI there is another possibility for the training of future teachers 
in loco who are taking the second half of undergraduate teacher formation courses, 
which is the Pedagogical Residency Program (RP). The RP is one of the actions that 
integrates the National Policy for Teacher Training and aims to induce the improvement 
of practical training in undergraduate courses, promoting the immersion of the student in 
the basic education school (CAPES, 2020). As in SI, in the RP the pre-service teachers 
perform observation and teaching activities. 

In a similar way to residencies in the health area, in RP the resident stays for a 
while in the place of his future performance as a professional, which, in this case, is the 
school, under the supervision of an effective teacher in the area he studies, the preceptor. 
However, with the pandemic caused by COVID-19, schools and universities were closed 
and the training activities that took place during SI and RP were hampered. In the follow- 
ing section, it will be presented how the teaching is offered today in Brazilian schools. 


The Teaching Offered Today in Schools 


In the state of Piaui, according to a survey carried out by the Piaui State Department 
of Education - SEDUC / PI (Piaui, 2020a), 88.82% of students enrolled in this network 
are having access to remote activities in a virtual way or by printed material. Among 
these students, access is distributed in different ways. As shown in Table 1, the teaching 
contact ranges from WhatsApp groups, instrumental platforms for organizing class 
content (Google Classroom), through other means, for example, phone calls, email, etc. 
11.18% of students in the state public network of Piaui still had no access to classes at 
the time of publication of the report (12/07/2020). 


Table 1 
Options for Remote Education Access in the Public Network of Piaui 





Contact WhatsApp Printed Ma- Google Class- 
Source Groups terial room 


Percentage (%) 42.40 24.49 12.52 9.41 11.18 
Source: (Piaui, 2020a) 


Others No access 





Of those students who attend classes, there is a predominance of the use of cell 
phone applications as a means of access. This shows that a large portion of the population 
has access to cellular devices and that they can also be used in Education. In view of 
this, at the end of November 2020, the Government of the State of Piaui made available, 
according to news on its official page (Piaui, 2020b), more than 180 thousand internet 
chips to students from the state network, to guarantee access to digital platforms that 
provide video lessons and activities produced by teachers, such as Canal Educagao, Preé- 
ENEM Seduc and the iSeduc Aluno application. 

One of the main consequences of remote education during the pandemic was 
the change from the school environment to the home environment. Such a change 
was not expected even by teachers who already adopted online environments in their 
teaching methodologies. This brought about such a rapid and emergency change, almost 
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obligatorily. At first, teachers and students had to adapt quickly to the use of digital 
platforms and new information and communication technologies (NICTs), which despite 
not being new, always faced challenges for their implementation, such as: the lack of 
investment in technologies in education; schools do not offer the minimum technological 
structure of pedagogical support to teachers and students; higher education courses that 
do not enable professionals to work using new technologies; in addition to the teachers' 
own resistance to work with these technologies (Soares-Leite & Nascimento-Ribeiro, 
2012). 

Teachers have become exposed to problems never experienced before (for the 
vast majority), 


“[...] for new problems, new answers, then the emergency to face COVID-19 has 
required updating and incorporation of new knowledge and technologies on the part of 
professionals who deal directly with suspects, confirmed cases and those who are in other 
care fronts. As well, remote work, distance education and telehealth, which were the 
targets of technical and ethical questions, become the means of maintaining services and 
attendance, permanent education, and social interaction (Falcao et al., 2020, p. 4, author's 
translation)”. 


In this way, teachers would need to become Youtubers by recording video- 
lessons and learning how to use video conferencing systems to adapt and develop new 
methodologies, to think about new ways of evaluating, and especially, to ensure the 
attention of students (Monteiro et al., 2012). 

It’s known the year 2020 was atypical, due to the occurrence of the pandemic, 
teaching underwent several transformations, causing the methodologies adopted within 
the classrooms to be modified. As a result, teacher training at a higher level to work in 
basic education had to undergo adaptations, starting from a face-to-face teaching model 
to an already existing model of Distance Education (DE), but little used by education 
professionals. This type of distance learning provided a basis for the remote education 
adopted in the pandemic period, which has brought a new form of interaction between 
students and knowledge and brought them closer to technologies, that is, it has brought 
new channels of interaction between teachers and students and, also, a new teaching- 
learning evaluation process. 

Thus, it was clear that part of education professionals was not prepared to take on 
remote learning, as the vast majority did not know how to use digital platforms or did 
not have the knowledge to handle them, due to the non-mandatory use in the classroom 
(Ribeiro Junior et al., 2020). Therefore, this lack of knowledge of many of the digital 
interfaces is due to the fact that subjects that teach and influence digital use in teaching are 
not offered in teacher formation courses. Another reason also for being associated with 
schools, for the most part, do not offer structure for teachers to apply digital teaching: 


“A survey launched in early April [2020] by the Innovation Center for Brazilian Education 
(Cieb) showed the basic education network's difficulty in finding solutions to face the 
Covid-19 pandemic. At the time of the study, 60% of the counties had no digital strategy to 
serve students during the period of social distance (Kochhann, 2020, author's translation)”. 
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According to an interview with the director-president of Cieb Lucia Dellagnelo, 
“in municipal networks, more than 70% had not used any online tool or methodology 
with their students. They saw that teachers do not know how to use these tools well” 
(Kochhann, 2020, author's translation). Corroborating this, in a survey conducted with 
52 teachers from different areas of the public and private education network in Piaui 
and Maranhao (another Brazilian FU, the second with the worst HDI) it was found that 
“T...] 52% of teachers have knowledge limitations in the use of educational technologies 
(text editing, internet research, video editing, among others), 27% need help from others, 
and 22% consider themselves self-sufficient” (Ribeiro Junior et al., 2020, author's 
translation). 

With face-to-face education not possible, the solution was to bet on digital 
platforms so that teaching would not be stopped. Platforms and applications such as 
Youtube, Google Classroom, Google Meet, Zoom, WhatsApp, OBS Studio, among 
others, were essential for teachers to transmit knowledge, trying to minimize the losses of 
the school year. However, the difficulties become even greater when part of the students 
do not have access to such platforms, due to the lack of conditions to have good quality 
internet or devices that can access them. 


“Thus, considering the social function of the school in the pre-pandemic and post- 
pandemic, it is important to understand the use of educational technologies for basic 
education has potentialized new learning for the teaching staff (mainly), although the focus 
has been on student, for whom there has been double attention in an attempt to cushion the 
effects of social isolation and ensure the teaching and learning of students (Ribeiro Junior 
et al., 2020, author's translation)”. 


Pandemic brought about a great transformation in the lives of teachers and students, 
leaving as legacy the immersion of remote education as the main tool for education 
networks, both private and public. In the post-pandemic, it is believed hybrid education, 
part with distance learning and the other face-to-face, will be part of the daily education, 
as the school will continue to have its role of learning and socialization, however mixed 
with new technologies. 

Given this, the following question came up: how to prepare future Physics 
teachers to carry out their teaching activities during supervised internship or pedagogical 
residency in basic education, in Teresina, Piaui during the period of social isolation? It 
is aimed, specifically, to understand how pre-service Physics teachers will be able to 
perform SI and RP activities remotely, acting in the way schools are functioning and 
helping to improve the performance and interest of students in the discipline of Physics 
in this pandemic period. Below, the methodology used to carry out the research. 


Research Methodology 
General Background 
This study is an exploratory and qualitative research. Exploratory because it seeks 
to examine and discover a still unknown reality. This method mainly uses qualitative 


research techniques based on observations and interviews (Selltiz et al., 1987). The 
data were produced by conversations with the supervising teachers (identifying how 


https://doi.org/10.33225/BalticSTE/2021.152 | | 











| Proceedings of the 4" International Baltic Symposium on Science and Technology Education, BalticSTE2021 


157 


the classes have occurred), proposing new activities and verifying the results of these 
activities through conversations and / or reports. 

According to Liebscher (1998), to learn qualitative methods it is necessary to 
learn to observe, record and analyze real interactions between people, and between 
people and systems. This research was carried out with pre-service Physics teachers, 
who were enrolled in the supervised internship disciplines of teaching physics, or they 
were participating in the RP program both at the Federal University of Piaui (UFPI), 
located in Teresina. This research included pre-service Physics teachers at the University 
and in-service Physics teachers at the secondary schools. The project was only possible 
with this interaction between the university and the school, so that the implementation 
of the SI or RP occurred in real schools and with the participation of Physics teachers as 
partners. All the activities occurred remotely. 


Sample Selection 


In general, in pre-service Physics teacher training courses, there are many dropouts 
and few finish the course. The supervised internship discipline is allocated at the end of 
the course and the participants of the RP need to be in the second half of the course to be 
able to participate in that program. In this research there were all the 20 enrolled students 
in the SI disciplines who were in teaching activities and 24 students who were involved 
in RP activities. 

All 44 pre-service teachers participating in the research carried out Physics teach- 
ing activities in public basic education schools in Teresina, under the supervision of the 
Physics teachers of the school, in the role of supervising teachers (for the 20 SI students) 
or preceptors (for the 24 RP students). Both groups were guided by the supervising pro- 
fessor, the author of this text. There were 3 schools that had RP students inserted and 8 
schools in which the interns were inserted. The pre-service teachers were asked if they 
authorize the use of their reports and speeches as a data source for a research and they 
authorized the use for this. 


Instruments and Procedures 


The research started through conversations with the supervising teachers and 
preceptors, in which they reported to the pre-service Physics teachers how the teaching 
was occurring. Based on this information and after contacting the school's classes, as 
observers, the pre-service teachers were able to propose new activities to be used in their 
classes. The verification of the results of these activities took place through conversation 
circles and / or reports. 

The data produced for the writing of this article was obtained through the oral 
reports of the pre-service teachers and / or the supervising professors and preceptors, 
which were recorded in a field diary, and through the reading of the reports produced by 
the pre-service teachers of the activities of the SI and the RP, too. 
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Data Analysis 


Considering these described materials, the corpus analysis was performed, using 
Bardin's content analysis (2016). The data analysis process itself involves several steps 
to give meaning to the collected data, which are organized by Bardin in three phases: 
1) pre-analysis, when the material is organized with the aim of making it operational; 
2) exploration of the material, when the definition of categories and identification of 
the registration and context units in the documents occurs; and 3) treatment of results, 
inference and interpretation, when condensation and highlighting of information for 
analysis occurs, generating moments of intuition, reflective and critical analysis. 


Research Results 


Based on the responses of the in-service teachers, the preceptors or supervising 
teachers, activities were proposed to be carried out by residents and interns. It was ver- 
ified in the three schools that functioned as RP nucleus the classes were taking place 
through Google Meet, in one school that received trainees the teaching was hybrid (face- 
to-face and remote, simultaneously, and alternating students), 2 used Google Meet and 5 
was working via WhatsApp. 

The students at the schools where the pre-service teachers acted were poor and a 
large part of them did not have access to quality internet or access by computers but used 
mobile data on smartphones. Because of this, most schools operated via WhatsApp, so 
that students could follow the activities. 

Based on these findings, the proposed activities would be carried out by interns 
and/or residents and the suggestions were categorized according to the type of resource 
or activity listed. The categories that emerged after the analysis of the field diary and the 
reports produced by the interns and residents were: Digital Platforms; Simulators - To 
Develop; Simulators - To Use; Gamification; and Others. 

In the Digital Platforms category, platforms that were being used in schools that 
had classes via lives (the 3 of the RP and 2 with interns) were mentioned and used these 
platforms to transmit the classes live. The following platforms were listed: RNP, Zoom 
and Google Meet. These three platforms were made available for free to use. RNP is a 
platform developed by the Brazilian government and all students and professors at feder- 
al universities who have corporate e-mails can open rooms and use them. This platform 
was used by only one resident, who opened the room and used it in the classes she taught. 
The other residents and interns used the platforms with rooms opened by the in-service 
teachers. 

Another category that emerged after analyzing the research corpus was Simula- 
tors - To Develop, in which Python, Scratch and Tracker were cited. These simulators 
are programmed by their users, differently from those will be commented on in the next 
category. They were only used in classes which occurred with lives through digital plat- 
forms. These simulators were used only in 2 schools, with Python and Tracker being 
used in one of them and Scratch in the other. Both schools were nuclei of the RP. 

In the Ist school, the one that used Python and Tracker, the use of these simula- 
tors was proposed by residents and in the 2nd school it was the tutor who used it and 
demonstrated to residents how to do it (this teacher finished his master's degree in Phys- 
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ics teaching, in which he used this resource). This fact was curious because it demon- 
strated the two-way street that occurs when the school inserts pre-service teachers in its 
activities: both the pre-service, in this case, the resident, and the in-service teacher, the 
preceptor, learn during the interaction. 

Simulators - To Use were verified in the reports of both residents and interns 
and these resources were used both in classes that took place via lives and in those that 
took place via WhatsApp. The following simulators were mentioned: Ludoteca, Fisica in 
Maos and PhET. These simulators are available ready for use, requiring only the inser- 
tion of numerical values or adjustments such as intensity, angulation, etc., according to 
the theme that was addressed in each one. They were used as an alternative to replace the 
experimental activity and demonstrated during the lives or proposed as complementary 
activities for students who had classes via WhatsApp. 

Gamification proposal was also used in classes which took place via lives and 
those took place via WhatsApp. As in the Simulators - To Develop category, activities 
in this category were proposed by both pre-service and in-service teachers. In two of RP 
schools, the preceptors suggested using this approach and explained to residents how 
these should occur. In two other schools, but which had interns, the interns who suggest- 
ed gamified activities. In these two schools, activities took place via WhatsApp. 

Among the tools used in the gamification process can be highlighted: Google 
Forms, kahoot, quizizz, quizlet and mentimeter. Google forms, kahoot, quizizz and qui- 
zlet were used to ask questions for teams, with kahoot, quizizz and quizlet showing the 
team that responded most quickly, being indicated for synchronous activities and Google 
Forms could also be used in asynchronous activities, in proposals such as, for example, 
gymkhanas. Mentimeter was used more as a graphic element, indicating the most fre- 
quent responses, assisting the teacher in the diagnosis of the class. 

Others category encompassed the resources cited by the interns who were work- 
ing in schools where classes were taught via WhatsApp and which could not be included 
in the previous categories. The following were mentioned: Youtube, social networks (in- 
stagram, TikTok and Facebook) and memes. As students using WhatsApp mostly used 
mobile data, the resources used by pre-service teachers had to consume the least amount 
of these, so that students could follow the other classes, too. 


Discussion 


Results presented above demonstrate some possibilities to conduct remote 
teaching. The five categories listed (Digital Platforms; Simulators - To Develop; 
Simulators - To Use; Gamification; and Others). The activities of Supervised Training 
and Pedagogical Residence occurred in schools where the students are poor and a large 
part of them did not have access to quality internet or access by computers but used 
mobile data on smartphones. The most classes take place via whatsapp. 

As there was no concern with teaching for the use of platforms in classes in a 
period before the pandemic, the pre-service teachers, although they already had remote 
classes at the university, learned to use the platforms with use in schools. Thus, it was 
clear that part of education professionals was not prepared to take on remote learning, as 
the vast majority did not know how to use digital platforms or did not have the knowl- 
edge to handle them, due to the non-mandatory use in the classroom (Ribeiro Junior et 
al., 2020). 
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It was commented by all pre-service teachers who used these tools that school 
students did not turn on their cameras and their active participation was limited to about 
2 or 3 per class. A fact that deserves to be highlighted is the supervising teachers and the 
preceptors commented that the students who participated in the remote classes were the 
same ones who were active in the face-to-face classes. The same was true for classes via 
WhatsApp. 

In the state of Piaui, according to a survey carried out by the Education Department 
of the State of Piaui - SEDUC/PI (Piaui, 2020a), 88.82% of students enrolled in this 
public network are having access to remote activities virtually or through printed 
material. Among these students, access is distributed in different ways. As shown in 
Table 1, the teaching contact ranges from WhatsApp groups, instrumental platforms for 
organizing the content of classes (Google Classroom) and even through other means 
such as, for example, phone calls, email, etc. 11.18% of public-school students in Piaui 
continued without access to classes at the time of publication of the report (12/07/2020). 


Table 1 
Options for Remote Learning Access in the Public Network of Piaui 








Contact source | WhatsApp groups _~ Printed Material poodle Clas Others miout 
room access 
Percentage (%) 42.40 24.49 12.52 9.41 11.18 





Source: (Piaui, 2020a) 


However, even teaching via Whatsapp was harmed because, according to a 
complaint by Souza (2021), ina May 2021 report, many students are still without internet 
or cell phone/tablet to carry out the activities after more than a year of the pandemic. This 
fact made remote teaching even more difficult and forced pre-service teachers to think of 
new proposals for this teaching. 

One of these proposals was by use of simulators. It was verifying two different 
groups of them: Simulators — to Develop and Simulators — to Use. The first group is more 
laborious to use because it needs programming. Tracker proved to be the simplest of the 
three (Python and Scratch were the others in this category), because a recorded video is 
inserted and some data is described so that the movement can be described and quantified 
(this simulator can be used only in Mechanics, for it serves to describe the movements). 
Python is widely used in mechanics, but it can be worked in other areas of Physics, too, 
such as in electromagnetism (Cristiano et al., 2019) and Scratch can be used for other 
areas of physics, too, and for production games and other purposes. The students really 
liked the use of these simulators, especially the Tracker, because videos recorded by the 
residents were used, showing themselves to be an instrument that brings Physics closer 
to the students' reality, as commented by Sung, Ma, Choi and Hong (2019). 

Both Simulators — to Develop and Simulators — to Use were used as an alternative 
to replace the experimental activity and demonstrated during the lives or proposed as 
complementary activities for students who had classes via WhatsApp. Very widespread 
in the teaching of Physics (Neri, Noguez et al., 2018), simulators have been used by 
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supervising teachers and preceptors since before the pandemic and are used mainly as 
evidence of theory taught in previous classes. 

Gamification proposal was also used in classes which took place via lives and 
those took place via WhatsApp. Gamification uses, outside the context of games, ele- 
ments of its design to retain the user's attention, motivate and increase activity (Deterd- 
ing et al., 2011). Among the elements of games that are used are: clear rules, objectives, 
rewards, inclusion of error in the process, immediate feedback, intrinsic motivation, ab- 
straction of reality, narrative, competition, levels, cooperation, conflict, voluntariness, 
etc. (Fardo, 2013). The gamified activities were other way used to encourage students to 
interact more and motivate them, since students have been quite unmotivated in remote 
activities (Barbosa, 2020). These activities occurred at two schools where the classes 
took place via WhatsApp. 

The last category, Others, included social networks and memes. The social net- 
works cited by pre-service teachers were: Youtube, instagram, TikTok and Facebook. It 
was found that students accessed social networks with great frequency, being therefore 
easy for students to see. Another factor led the trainees to propose these resources was 
the fact that access to many social networks and Youtube are outside the consumption of 
the students' internet franchise, which they can access without running the risk of ending 
it. 

Most of the time the pre-service teachers forwarded the links to videos or memes 
before classes, in order to generate curiosity in the students. Some of them questioned 
the videos and/or memes in class, but even so, feedback was not as expressive, although 
greater than without the use of these resources. Memes were funny, according to student 
comments, allowing pre-service teachers to comment in more depth on the phenomenon 
addressed, as commented by Baysac (2017). 

It is also interesting that different approaches have been proposed for use on 
WhatsApp. The standard form used by in-service teachers was to send activities before 
class and, at a specified time, in-service teacher was online to answer questions and nor- 
mally no students showed up for this purpose. Then, a pre-service suggested that along 
with the materials sent (slides and text in pdf) a brief explanation, typed in WhatsApp 
itself was sent. Another intern likewise wanted to send an explanation of the content, 
but used the format of podcast, with short audio recording on WhatsApp itself and said 
the students' understanding improved a lot, since they were too lazy to read and had no 
initiative to question. 

Since, according to a survey carried out in January 2021, about 4 million students 
dropped out of school as a result of the pandemic, with the dropout rate being higher 
among people from the poorest social strata, such as in classes D and E, in which the 
dropout is 10.6%, while among class A people it is 6.9% (CNN, 2021). Although it is 
not possible to quantify it, what it was seen by the investigated schools is this number 
is much higher, in the range of 30% of dropout students. So, all proposals mentioned in 
the five categories mentioned above were made with the aim of making the classes more 
dynamic and achieving greater interaction with students, as well as helping to reduce 
school dropout rates. 
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Conclusions and Implications 


The internship period and the Pedagogical Residency proved to be a rich learning 
period for the pre-service teachers. Even though the activities took place remotely, the 
pre-service teachers were able to use different resources and methodologies, experiencing 
relevant experiences for their training as future teachers, inserted in the reality of the 
current context. 

The pre-service teachers, both SI and RP, proposed different activities in relation 
to those offered by in-service teachers and helped to improve the quality of education 
offered to students from public schools in Teresina. It was found that remote / hybrid 
teaching has a wide range of possibilities for teaching action, but learning is needed so 
that NICTs can be widely used. 

The supervised internships and the Pedagogical Residency proved to be important 
so that future pre-service teachers could experience in-depth the contents learned during 
initial training, this applied in real teaching situations. And, in addition, the pre-service 
teachers, by proposing varied activities during the RP and / or SI, were able to assist in 
the recycling / training of in-service teachers and in the motivation of their students. 

New research must be carried out in order to provide a higher quality in the 
actions of Pedagogical Residency and supervised internships, as well as to integrate in 
an increasingly harmonious way the university and the school, collaborating so that a 
higher level education is offered in schools. 
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Abstract 


STEM education has increasingly drawn attention internationally in recent years. In Malaysia, 
efforts to encourage students to take up STEM subjects have risen, but student enrolments in 
almost every STEM subject area have continued to fall over the last decade. The situation is 
even more challenging in Sabah, an East Malaysian state where 72 percent of its schools are 
located in rural areas with basic utilities and limited infrastructures. Therefore, a STEM Mentor- 
Mentee outreach program through university-school partnership was developed to address 
the gap in STEM education attainment. The program targeted tenth graders (aged 16 years) 
from rural secondary schools to help them learn STEM by relating it explicitly to their local 
environment. STEM activities were guided by the engineering design process while harnessing 
their 21st century skills. Mentors consisting of in-seervice and pre-service teachers who provided 
guidance, support and assistance to mentees. Data were captured through mentees’ responses to 
open-ended questions, mentors ’field notes, focus group observation and interviews. A total of 732 
students, 342 in-service and 99 pre-service teachers were involved in the programs from 2015 to 
2019. Findings suggest that the program was able to develop creativity, problem solving, critical 
thinking and teamwork skills among rural secondary school students. 

Keywords: 2/st century skills, mentor-mentee, outreach program, rural schools, STEM 
Education, university-school partnership 


Introduction 


STEM (Science, Technology, Engineering, Mathematics) education has been 
given priority in many countries to produce a young generation that is able to compete 
in the 21st century. The integration of the disciplines of knowledge consisting of science 
(Physics, chemistry, and biology) and mathematics with technology and engineering into 
one field of education is known as STEM Education (Ministry of Science, Technology 
& Innovation, 2015). In Malaysian Education Development Plan (PPPM) 2013-2025, 
STEM is mentioned explicitly as a specific initiative to be taken by the Ministry of 
Education Malaysia (MOE). The initiative is to strengthen STEM education as to produce 
human capital in the 21st century who have higher order thinking skills, innovative, 
prudent, independent, technologically literate, able to create, able to solve problems and 
make decisions (MOE, 2016). 

The demand for a STEM driven workforce in Malaysia has become a burgeoning 
need as the economy has evolved from a production-based economy to a knowledge- 
based economy. It has been estimated that Malaysia would need 500,000 scientists 
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and engineers by 2020 to cope with the challenges of the Fourth Industrial Revolution 
(Academy of Sciences Malaysia, 2015). However, at that point, it had only 70,000 
registered engineers. Undeniably the supply of STEM related workforce is highly 
dependent on new entrants into STEM related program in upper secondary as well as 
tertiary level. However, report has shown that only a total of 22.5% of students have 
enrolled in science stream, and technical and vocational secondary school classes in 
2017, which is still far from the ideal ratio of 60:40 Science/Technical: Arts Policy set in 
1970 (Academy of Sciences Malaysia, 2018). 

The challenge of achieving the 60:40 Science/Technical: Arts Policy is even 
tougher for the vast rural areas of Malaysia due to its limited infrastructure, lack of good 
schools and small population (Ling et al., 2015). Sabah, an East Malaysian state with 
a relatively high proportion of students in rural schools is facing a more challenging 
situation with respect to its efforts to keep pace with STEM Education. According to 
the Sabah Economic Development and Investment Authority Blueprint (SDC, 2011), 
72% of Sabah’s schools were located in rural areas. In terms of infrastructure and basic 
utilities, most rural primary and secondary schools in Sabah lack supplies of 24-hour 
electrical connection, access to good teaching and learning resources, computers, and 
science laboratories. It is apparent that these limited opportunities and facilities have 
somewhat created a gap in STEM education attainment between rural and urban schools 
in Sabah and in Malaysia as a whole. 

In countries such as Colombia and the United States of America, an outreach 
program is usually designed to help and encourage disadvantaged students of rural 
schools to increase their STEM literacy and enthusiasm. These after school STEM 
outreach programs aim to improve the quality of science education (Laursen et al., 2012), 
motivate school students to choose STEM subjects in the future, and generate more 
graduates who have the capacity to pursue science-based careers (Moskal & Skokan, 
2011; Office of the Chief Scientist, 2013). While these outreach programs examined 
the students’ STEM literacy and enthusiasm, there is a lack of research examining the 
program impact on students 21st century skills, thus a gap in the literature that this 
research will fill. 

Brookshire (2014) highlighted the guidance from the right mentor in mentoring 
can expand students’ ideas about the possible careers in STEM fields and trigger a 
passion for STEM. These observations raised a crucial question: “How would a mentor- 
mentee outreach program help students in rural secondary schools learn STEM?”. 
Tackling questions like this, particularly in rural settings often requires a more integrated 
approach to STEM education. 

According to Essex (2001), school-college partnerships hold significant promise 
for renewal and improvement in education (pp. 736). Essex pointed out that successful 
partnerships allow both the school and the university to work together in an environment 
in which synergy leads to better decision making, thus having a positive effect on student 
learning. In this matter, a school-university partnership can help to address the needs 
of rural schools by providing adequate support and access in the development and 
delivery of STEM outreach program. Thus, there is a need to adopt an integrated school- 
university partnership and mentoring approach to produce maximum mutual benefit 
for all involved in the outreach program, and to seamlessly examine what rural school 
students would learn during the outreach program. By exposing children to STEM and 
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giving them opportunities to explore STEM-related concepts, they will be equipped with 
21st century skills such as critical thinking, problem solving, creativity, collaboration, 
and communication skills as desired by MOE Malaysia (MOE, 2016). 

To ensure inclusive and equitable STEM education, a STEM mentor-mentee 
outreach program through university-school partnership approach has been developed 
to support disadvantaged and marginalized students in the rural areas. Specifically, this 
outreach program aims to improve the reach of STEM education in schools geographically 
distant from the cities of opportunity based on the concept of contextualization and 
collaboration. This enables rural students to understand STEM by relating it explicitly to 
their local environment and to increase their 21st Century skills. As such, the objective 
of STEM mentor-mentee outreach program is to find out if mentor-mentee outreach 
program would help students in rural secondary schools develop their 21st century skills 
related to STEM. 


The University-School Partnership Model 


Bernay (2020) proposed Transformational Learning Community (TLC) model as 
one form of school—university partnership to address the challenges of the future. The 
TLC model suggests that the current teaching practices in schools and in universities 
should be extended into the real world. The purpose is to generate thinking and 
knowledge creation for children or student teachers to thrive in today’s world. Bernay 
continued to address that learning communities should focus on change; to reinvent 
schools for transformational learning for children, student teachers, teachers, university 
lecturers and the wider community. Recognizing the advantages of TLC model to cater 
to the needs and challenges faced by students in the rural areas, TLC model was adapted 
in realizing the school—university partnership in the outreach mentor-mentee program. 
Figure | illustrates the range of relationships and resulting benefits in the adapted TLC 
model. 


Figure 1 
Transformational Learning Community Model (adapted from Bernay, 2020) 
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In order to work in an authentic partnership with rural schools in Sabah, a 
community of practice was created to draw on the expertise and experience of university 
science lecturers and student teachers involving in-service and pre-service teachers and 
co-constructing outreach programs with school teachers, principals and parents. In other 
words, university Malaysia Sabah (UMS), UMS science lecturer, UMS in-service and 
pre-service teachers, heads of school science department, school principals and parents 
were working as a team to carry out the program. The parents granted permission to 
allow their children to participate during the weekend (Saturday). School principals and 
heads of science department collaborated to provide the school hall and PA system to be 
used for the activities. The university provided free bus service to transport the mentors 
and materials to the schools which are located 16-180 kilometers from Kota Kinabalu, 
Sabah. 

The resources used for running the program were funded by schools and university. 
The Centre for External Education of the university allocated RM50 to each of its master 
students to run the program outside the classroom, which was used to support the cost 
needed to buy materials and equipment. The in-service teachers also sought sponsorship 
from the local community such as businessmen to support the cost for food and drink 
for mentees and mentors. The in-service teachers also play their roles in borrowing 
science apparatus from their school laboratory such as crocodile-clips, hot gun, and 
glue gun sticks to support the activities. Recycled materials such as plastic bottles, empty 
boxes, newspapers, and egg trays were collected by mentors which were used as main 
resources for the activities. Some tools and materials such as scissors, rulers and cutters 
could be re-used in the followed-up program. Administrative costs were extremely 
low as free messaging platform such as telegram and WhatsApp were used to ease the 
communication among people one month prior to the start of the program. 


Engineering Design Process Model 


The Massachusetts Department of Education (2006, p. 84) proposed eight steps 
of engineering design process which provide a guide for teachers and curriculum 
coordinators regarding learning, teaching, and assessment in science and technology/ 
engineering specific content from Pre-Kindergarten to Grades 6-8 and throughout high 
school. Those eight steps of engineering design process include identifying the need 
or problem, research the need or problem, develop possible solution(s), select the best 
possible solution(s), construct a prototype, test and evaluate the solution(s), communicate 
the solution(s), and redesign. 

Many researchers claim that STEM curricula can be integrated in an engineering 
design process to provide a mechanism through which students learn relevant STEM 
content (Hmelo et al., 2000; Mehalik et al., 2008; Schunn, 2009). This mechanism 
encourages students to make connections, helps connect design failure or next steps to 
real world engineering and technology (Lottero-Perdue, 2015). Students learn important 
scientific concepts and their application in engineering and technology, as well as their 
relationship and application in daily life or real-world context. Students could look for 
connections by engaging with activities or material in 'real-world' contexts to establish 
relevance. This approach can attract students’ interest in science lessons and provide 
them with a deep understanding of concepts and meaningful learning. 
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Hynes et al. (2011) noted that engineering design process that focus on solutions 
and construction of prototypes impel students to encounter the process of creative and 
critical thinking as well as problem solving skills. Hence, engineering design process 
would offer a route as an instructional framework for fostering 21st century skills among 
rural secondary school students. According to Mentzer, Huffman and Thayer (2014), 
engineering design problems embedded in the context of an engineering design process 
in practice tend to be structurally open-ended and highly complex. Engineering design 
problems are also designed to be ill-defined. King and Kitchener (1994) characterized 
ill-defined problems as problems that have no clear-cut right answer and require the 
integration of many skills and abilities. Solving ill-defined problems requires judgment, 
planning, the use of strategies, and the implementation of previously learned skill 
repertoires. 

Science teachers in previous program noted several potential challenges while 
implementing a STEM-project-based learning approach in their rural school classrooms 
(Siew et al., 2015). These included inadequate materials, limited facilities, and limited 
allocation of classroom time. Accordingly, the engineering design process employed in 
this program removed the ‘redesign’ step proposed by the Massachusetts Department 
of Education (2006, p. 84). This modification was made to ensure that students could 
produce workable prototypes that made best use of the materials and time provided in 
the program. 


The Design and Development of STEM Mentor-Mentee Outreach Program 


The STEM mentor-mentee outreach program was designed and developed by 
a science lecturer as program coordinator and assisted by in-service and pre-service 
teachers. The pre-service teachers were second and third-year undergraduates aged 
between 22 and 23, training in Physics and Mathematics Education and had no teaching 
experience in schools. In-service teachers were qualified teachers with degrees in Science 
and Social Science Education who were undertaking Master course at the time. 
Mentor-Mentee. Prior to the program, the faculty recruited mentors among in-service 
and pre-service teachers, each of whom signs up voluntarily in exchange for experience 
and competence. Mentors have attended a one-day training course in peer mentoring 
arranged by the program coordinator. Mentors were trained to conduct facilitation and 
assessment to their mentees. The mentors were empowered to carry out their role as 
evaluators and facilitators during the program. After completion of their mentor role, 
mentors are awarded a Certificate of Contribution by the faculty dean. 

The Grade Ten Science Stream students from the secondary rural schools were 

chosen as the mentees of the program. Mentees received guidance, support and assistance 
from mentors in finding solutions to problems, using materials, sketching, and building 
prototypes. 
Ill-defined Problem and STEM Activities. The ill-defined problem was introduced to 
students within the context of their daily life. Students were engaged to connect their 
daily life experience to solving ill-defined problems. Students would be also asked to 
consider the constraints of the materials and time, think about what they already know, 
design, plan, construct, test and evaluate a physical prototype of their design. 
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STEM activities were guided by the adapted engineering design process (EDP). 
The EDP provides a flexible process that takes mentees from identifying a problem, 
designing a solution to developing, creating, testing and evaluating a prototype to solve 
daily life problems in their environment using inexpensive materials. It allows students 
to realize that there are many ways to find solutions, as they engage in brainstorming 
to identify problems and propose solutions. The process of finding the optimal solution 
based on constraints requires participants to engage in critical thinking, creativity, 
imagination, collaboration and communication skills, and problem-solving skills. 

Different STEM activities were introduced in each school with local context 
to enhance learning and understanding of the STEM concepts. A total of three STEM 
activities were introduced in each school, each taking about three hours and 20 minutes. 
Examples of ill-defined problems embedded in STEM activities are as follows: 


1. “Ali saw a bird perched on a tree branch. A question arose in Abu’s mind, ‘How 
can it perch for such a long period of time?’ 


Your task: Create a balancing toy that can stand stably on your fingers like a bird. 
Each student must produce at least one balancing toy that meets the requirements. 


2. “After a year of construction, the Mesilou river bridge that collapsed due to 
the earthquake has been completed. This news is quite exciting for the villagers who 
want to cross the Mesilou river. But some are wondering: How much weight can the 
vehicle support by the bridge? Is it really safe? You and your friends have been given 
the opportunity to show the villagers that the Mesilou bridge is actually strong enough to 
support the weight of vehicles crossing it”. 


Your task: You and your friend are asked to prove it by designing and building a 
bridge that crosses a river which is one meter wide. The bridge is strong enough to hold 
at least 3 cans of coca cola. 


Higher-Order Thinking (HOT) Questions. Students also needed to answer the 
Higher-Order Thinking (HOT) questions that stood of questions that were not strictly 
in their curriculum. In a way, answering HOT questions inspired students to acquire 
new found competences. Anderson and Krathwohl (2001)’s Taxonomy was used as a 
guide to develop a blueprint for the HOT questions, which belonged to the Analysis and 
Evaluation category of the cognitive domain. Some samples of HOT questions used 
were: ‘In your opinion, if buildings were constructed identical to this prototype, is it safe 
to be inhabited? If yes/no, please explain why?’(Evaluation); ‘How can your prototype 
be modified in order to improve its results in the future?’(Analysis); and ‘Explain why 
there is a difference of the submarines’ speeds between the two bottles? (Analysis)’. The 
HOT questions were specially designed to evaluate students’ critical thinking skills in 
connecting STEM activities with their daily life. 

Scoring Rubrics. Scoring rubrics were developed for mentors to evaluate the 
prototypes produced by the group during the presentation and testing. Aspects assessed 
were product functionality, sketches, group collaboration, and understanding and 
application of scientific concepts. Scoring rubrics were constructed based on analytical 
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scoring. The quality of student responses and products was assessed from “Poor” (lowest 
level) to “Very Good” (highest level). 


Research Methodology 
Research Design 


A single group with intervening STEM mentor-mentee outreach program design 
was used in this research. This was a case study whereby qualitative approach was 
employed to gain an in-depth knowledge of the participants’ 21st century learning 
experiences in implementing STEM mentor-mentee outreach program. The Grade Ten 
Science Stream students (aged 16 years) from the secondary rural schools were chosen 
as the participants of the research. 


Research Instrument 


Research instruments included mentors’ field notes and mentees’ responses to 
open-ended questions; focus group observation and interviews. 

Field Notes. Mentors wrote their field notes based on the observation made during 
the STEM activities, and the semi-structured focus group interviews with mentees. 
Focus group observations were collected using an observation form adopted from 
scoring guides developed by Wang et al. (2015). The quality of the students’ responses 
was ranked from “0” (Level 0: the lowest level) to “3” (Level 3: the highest level). 
The interview questions were open-ended and the students were encouraged to draw 
explicitly from their learning experiences of working on the STEM activities. Each focus 
group interview was conducted in groups consisting of 4-5 mentees after the completion 
of each STEM activity. 

Open-ended Question. A paper-based open question was administered at the 
end of the program. in view of obtaining written feedback from mentees Students were 
asked to reflect on their learning experiences following their participation in the STEM 
mentor-mentee program by responding to the question: ‘Something new I have learned 
today was...” The open-ended questions offered the mentees an opportunity to clarify, 
and to fill gaps of missing information not captured by the focus group observations and 
interviews with mentees. 


The Implementation of STEM Mentor-Mentee Program 


The program was implemented in 16 rural secondary schools in the districts of 
Tenom, Tambunan, Ranau, Tuaran, Kota Marudu, Kudat, Penampang, Putatan, Telipok, 
Sipitang and Kiulu throughout 2015 - 2019. Throughout the five-year program, a total of 
732 students, 342 in-service and 99 pre-service teachers were involved in STEM mentor- 
mentee outreach programs. 

In these programs, mentees in group of four or five collaborated to solve an 
ill-defined problem utilizing the engineering design process. Mentees worked 
collaboratively to design, build, and test prototypes of their inventions according to the 
criteria set in an ill-defined problem. Mentees could respond to the ill-defined problems 
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in many ways with many different solutions. Mentees also would have opportunities to 
make use of their STEM knowledge and skills to solve the problem and choose their 
favorite strategies to obtain their unique solutions. Aspects of science and mathematical 
concepts and communication skills were also emphasized and evaluated during the 
presentation of prototypes by group members to their peers and mentors. 

Group mentoring involves one or two adult mentors forming a relationship with 
mentees. The mentors play the role of facilitator and makes a commitment to interact 
with the mentees over the period of one-day program. The interaction is guided by 
activity worksheets, which allow time for two ways discussion of the STEM activities. 


Data Analysis 


Mentors’ field notes and mentees’ responses to open-ended questions were 
analyzed using thematic analysis. Thematic analysis is a form of a pattern recognition 
technique by searching through the data for emerging themes. 


Research Results 
Solving Daily Life Problems using Scientific Knowledge 


Almost every mentee (98%) noted that they benefitted from the STEM activities 
as they were exposed to real-life situations where scientific knowledge was applied for 
solving daily life problems. More importantly, STEM activities succeeded in providing a 
platform for them to apply scientific knowledge in solving problems. Among the scientific 
concepts the mentees noted were related to water and air pressure, equilibrium of force, 
base area, balanced force, surface tension, stability, water density and the buoyancy 
force in a submarine. Mentors from one group confirmed that interviews with mentees 
revealed that mentees found the need to apply the concept of impulse in order to create 
an innovation that help to absorb the impact of an egg being dropped from a high place. 


Answering Higher Order Thinking (HOT) Questions using Scientific Knowledge 


A significant number of mentors (88%) observed that a profound comprehension 
of scientific knowledge helped participants answer high level questions and to be creative 
in reapplying knowledge learnt in the designing and producing of prototypes. Mentors 
observed that HOT questions provide mentees an opportunity to think critically about 
the answers and make connections with scientific concepts they have learnt in class. 


Connecting STEM Activities with Daily Lives and Scientific Concepts Learned 


A large percentage (93%) of the mentors noted in their field notes that participants 
learned how to make connection of the STEM activities with their daily life phenomenon. 
For example, mentors observed that mentees could relate how ships or boats function 
and why they could float on the surface of water by making comparisons with their own 
made boat models. Another example is when answering the HOT questions, mentees 
could affiliate the floating needle and paper clip activity with the water strider bug, 
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a floating log, water lilies, floating ants, and others. Mentors supported these claims 
by noting that “Scientific knowledge is not only for answering exam papers but also 
useful in helping students create connections and explain situations faced in their daily 
lives. In this case, it is observed that students applied scientific concepts they learned 
during Physics lessons in problems given to them. Students not only applied the science 
principles and laws they learnt but also used them in practical forms’’. 


HOT Questions sparked Critical Thinking 


A large percentage (93%) of the mentees expressed through the open questions 
that they were challenged to think critically when answering the HOT questions in the 
STEM program. The mentees felt that the HOT questions were difficult, but they tried 
their best to answer and associate them with their prior knowledge. 

According to mentors, mentees were capable of giving rational answers to the 
HOT questions. For example, one of the group members gave an excellent answer and 
showed that he/she understood the concept and was able to give a suggestion to improve 
the existing prototype if given the chance to design it with the aid of extra materials. The 
sharing of answers added knowledge collectively to the group besides increasing their 
critical thinking skills. According to mentors, the STEM activities tested and challenged 
mentees to think outside the box using higher order thinking skills. 


Designing and Building Something New and Practical 


A large percentage (96%) of the mentees expressed in the open questions that 
STEM activities gave them an opportunity to create many new, interesting and practical 
science products using everyday materials. They stated that the balloon powered car 
made from plastic bottles was a new experience for them. They were fascinated with 
finding new ways to make a highly powered car moved by air using ever ready materials 
such as glue, bottles, pencils, and others. Another activity was making a boat. The 
mentees said they realized that play dough can float when shaped into a boat. Others 
noted that finding gravity center through making the balancing toys was a new activity. 
Meanwhile, a few mentees commented that they discovered new ways of floating the 
needles and paper clips. 

When participants were asked why they were excited with the STEM activity, 
they answered that: “because we got the chance to design and build a new model which 
we only see in textbooks”. Other than that, students showed interest in STEM activity 
because they could become ‘designers’ of their own boat in the future. Mentors observed 
that the mentees could design egg protection tools and that every group member worked 
together the whole time by contributing ideas and carrying out the projects as they had 
planned. Thus, according to mentors, STEM activity seems to provide a very good start 
to stimulate the interest of mentees in learning STEM. 


Thinking Creatively through Combination of Ideas 


A considerably large percentage (78%) of the mentors noted in their field notes 
that they were amazed when mentees exhibited creativity above anticipations. Group 
works enhance mentees’ ability to produce all kinds of products using limited materials 
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as a result of the combination of ideas by group members. This can be seen during the 
construction of the bridge using the newspaper. Many new ideas and views were brought 
up during brainstorming within the group. With a combination of ideas from group 
members, they were finally able to build a paper bridge that could be crossed by 3 cans 
of drinks. Thus, mentors noted that teachers need to acknowledge students’ potential and 
use the right tools and mentorship to enhance their hidden potentials. 


Ill-Defined Problems inspired Creativity and Thinking 


A significant number of mentors (93%) reported that mentees faced complexities 
posed by ill-defined problems in the program. These ill-defined problems demanded 
from them effective response to the challenging tasks which in turn inspired creativity 
and thinking. For example, the mentees’ creativity levels were tested while creating a 
bottle car that was powered by a balloon. Participants had to figure out ways to move 
a car by using only air within a balloon, and to think of a method of reducing the car’s 
weight and decrease its tire resistance 


Sketching, Designing and Constructing Models fostered Creative Thinking and 
Problem-Solving Skills 


A considerably large percentage (88%) of the mentees expressed opinions that 
STEM activity encouraged creative thinking, and also problem-solving skills. This 
was because each activity needed them to sketch and design models according to 
the creativity of each group. Mentees noted that they had to think of a way to design 
models that worked and at the same time possessed creative elements. For example, the 
activity of ‘balancing toys’ had successfully induced creativity and imagination within 
mentees as almost every one of them were able to build a balancing toy with different 
designs. Unexpectedly, mentees in one group were able to create nine balancing toys 
with different designs. Furthermore, this activity also enhanced mentees’ thinking skills. 
Mentees gained ideas on how to create their own toy design. Hence, it encouraged them 
to think more profoundly. This was supported by observations made by mentors who 
noted that: “Besides creating one ‘balancing toy’, students can think of ways to merge a 


d 


few ‘balancing toys’ in a stable condition”. 
Instilled Thinking through Teamwork 


A significant number of mentors (93%) observed that mentees not only came 
up with some thoughtful ideas but also showed a spirit of teamwork during the STEM 
activities. Mentors described mutual understanding and cooperation, boosted the 
confidence of each mentee to do his/her best work in order to construct a workable 
prototype. Mentees made attempts to cooperate as a group to solve the problems and 
brainstorming within the group helped mentees to increase their critical and creative 
thinking. This was proven when a group of five was able to create nine balancing toys 
with different designs. In another group, group members divided up some tasks such as 
rolling up a newspaper and stitching together some newspapers in order to produce a 
paper bridge which in their opinion was a very difficult and challenging activity. 
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Solving Problems with a Determined Effort 


A large percentage (92%) of the mentors reported that STEM activities challenged 
the mentees to think of many ideas and make many attempts without giving up. Mentees 
tested their prototypes through many attempts, improved or modified their original idea 
through the process of trial-and-error. For example, mentees made modifications to the 
boat several times so that the boat they built could hold up to 26 marbles. They proved 
their seriousness in solving the problem despite having to repeatedly test the boat's 
ability to accommodate large quantities of marbles. 


Discussion 


The STEM Mentor-Mentee outreach program executed a new idea which addresses 
a specific challenge of revitalizing STEM movement in rural secondary schools and adds 
value to rural school students, teachers, and the University. The program used university- 
school partnership model to reach out to rural schools to improve the 21st century skills 
and make it more relevant to local context. 

Evidence from mentees suggests that the program enabled them to apply STEM 
knowledge in solving daily life problems, designing and producing daily life products, 
and answering HOT questions. Mentees were also able to connect the STEM activities 
with daily lives and scientific concepts learned in the classroom, and to create new 
products using everyday materials. These findings make clear that the execution of the 
engineering design process in STEM mentor-mentee outreach program can help mentees 
in relating and applying STEM knowledge to their daily life problems and contexts. 
Researchers are in agreement that engineering design process provides a mechanism 
through which mentees learn to make connections by engaging in ‘real-world’ problems 
and contexts (Lottero-Perdue, 2015; Neo, Neo & Tan, 2012). 

The STEM mentor-mentee outreach program not only allowed mentees to gain 
and integrate STEM knowledge but also provided an avenue to boost their creativity, 
critical thinking, problem solving skills and teamwork. Mentees’ creative and critical 
thinking was sparked through solving HOT questions and ill-defined problems posed in 
the STEM activities. In addition, as mentees went through the engineering design process 
such as selecting the best solution in sketching, designing and constructing a prototype 
have helped them to foster their critical thinking, creative thinking and problem-solving 
skills. Hence, engineering design process offers a route for mentors to foster 21st century 
skills among mentees from rural secondary schools. Hynes et al. (2011) supported that 
engineering design process provides students an opportunity to practice critical thinking 
skills as well as outside-the-box thinking. King and Kitchener (1994) also asserted that 
exposure to ill-defined problems that mimic those solved by real-world practitioners help 
students develop problem-solving and critical-thinking skills. This research demonstrates 
that engineering design process employed in STEM mentor-mentee outreach program 
allows mentees to focus on solutions to ill-defined problems that could encounter them 
in the process of creative and critical thinking, and problem-solving skills. The findings 
of this research suggest that designing ill-defined problem scenarios for the program 
provide a framework by which mentees can engage in 21st Century skills. 
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In this STEM mentor-mentee outreach program, new ideas were generated 
through the combination of ideas of group members as well as through the trial-and- 
error method. Mentees as a group respond effectively even with limited materials and 
time in organizing their thoughts to choose the best possible solution for their prototype 
using related scientific concepts. Thus, this explains mentees who were actively engaged 
in collaborative activities could encourage their critical thinking development if mentors 
guide their critical thinking processes as addressed by Snyder and Snyder (2008). This 
research highlights the important role of mentors to boost the team spirit during the 
STEM activities. 

Similarly, in the STEM activities, students could respond to the ill-defined 
problems in many ways with many different solutions. Ill-defined problems engage 
students to make comprehensive use of their STEM knowledge and skills to solve the 
problem and choose different strategies to obtain their unique solutions. This is in fact 
a supportive environment in developing many unique responses through teamwork, in 
line with Sawyer (2003) who stated that people who are engaged in activities together 
produce a novel outcome. This research highlights that mentees who were engaged in 
group work during mentor-mentee outreach program were able to generate more novel 
solutions to a problem, hence promote their creative thinking. 

While the mentees described many positive learning experiences gained in this 
program, they also pointed out several challenges that test their resilience. The most 
commonly mentioned challenge was the conducted STEM activities required the use 
of a wide range of cognitive abilities. Mentees tested their prototypes through many 
attempts, improved or modified their original idea through the process of trial-and- 
error. This program makes clear that guidance, support and assistance by mentors would 
clearly help some mentees to complete the prototypes, and thus adequate mentoring 
from mentors to trigger students’ thinking is a critical matter and should be enforced by 
mentors. 


Conclusions 


This research provides support for the need of university-school partnerships in 
making STEM mentor-mentee outreach program a reality in the long run by offering 
a meaningful way of developing 21st century skills among rural high school students. 
This research exhibits that many 21st century skills involving high-level cognitive and 
transferable skills could be inculcated through one STEM mentor-mentee outreach 
program: problem solving, critical thinking, creativity, communication skills, and 
teamwork. These are the soft skills essential to be successful in a STEM career. 

This research also supports new research examining the potential implementation 
of STEM outreach programs and university-school partnership in elementary rural 
schools. Life skills such as entrepreneurial thinking skills can be introduced in the 
program to examine whether the rural school children could apply these skills in 
addressing "localized" problems and issues and turning problems into marketable 
product. Thus, the idea of STEM mentor-mentee outreach program and its activities can 
be adjusted and fine-tuned by any educators to be researched to new group of students 
based on the local context. 
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